-- Logs begin at Sat 2022-09-10 13:35:07 UTC, end at Sat 2022-09-10 14:37:10 UTC. -- Sep 10 13:59:11.685687 dv-cinder-856901-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 10 13:59:11.692974 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 10 13:59:11.695125 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sat Sep 10 13:59:11 2022] *** Sep 10 13:59:11.695125 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 10 13:59:11.695125 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 10 13:59:11.695125 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: nodename: dv-cinder-856901-1-iscsi Sep 10 13:59:11.695125 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: machine: x86_64 Sep 10 13:59:11.695125 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: clock source: unix Sep 10 13:59:11.695125 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: pcre jit disabled Sep 10 13:59:11.695621 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: detected number of CPU cores: 4 Sep 10 13:59:11.695621 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: current working directory: / Sep 10 13:59:11.695621 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: detected binary path: /usr/bin/uwsgi-core Sep 10 13:59:11.695621 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: your processes number limit is 47680 Sep 10 13:59:11.695621 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: your memory page size is 4096 bytes Sep 10 13:59:11.695621 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: detected max file descriptor number: 2048 Sep 10 13:59:11.695621 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: lock engine: pthread robust mutexes Sep 10 13:59:11.695621 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: thunder lock: enabled Sep 10 13:59:11.696548 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 10 13:59:11.696652 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 10 13:59:11.736282 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: Python main interpreter initialized at 0x55e73ba1c9c0 Sep 10 13:59:11.736282 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: python threads support enabled Sep 10 13:59:11.736282 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: your server socket listen backlog is limited to 100 connections Sep 10 13:59:11.736282 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: your mercy for graceful operations on workers is 90 seconds Sep 10 13:59:11.736674 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: mapped 403077 bytes (393 KB) for 2 cores Sep 10 13:59:11.736754 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: *** Operational MODE: preforking *** Sep 10 13:59:11.737002 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: *** uWSGI is running in multiple interpreter mode *** Sep 10 13:59:11.737002 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: spawned uWSGI master process (pid: 81575) Sep 10 13:59:11.737206 dv-cinder-856901-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 10 13:59:11.738570 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: spawned uWSGI worker 1 (pid: 81588, cores: 1) Sep 10 13:59:11.738570 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: spawned uWSGI worker 2 (pid: 81589, cores: 1) Sep 10 13:59:11.738570 dv-cinder-856901-1-iscsi devstack@keystone.service[81575]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 10 13:59:16.049185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 13:59:16.049185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: from cryptography.utils import int_from_bytes Sep 10 13:59:16.049966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 13:59:16.049966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: from cryptography.utils import int_from_bytes Sep 10 13:59:16.051038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 13:59:16.051038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: from cryptography.utils import int_from_bytes Sep 10 13:59:16.051508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 10 13:59:16.051508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: from cryptography.utils import int_from_bytes Sep 10 13:59:16.101317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 10 13:59:16.101487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 10 13:59:16.101680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] command line args: [] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 10 13:59:16.101949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 10 13:59:16.102127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ================================================================================ {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 10 13:59:16.102381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] admin_token = **** {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.102459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 10 13:59:16.102619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 10 13:59:16.102697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] config_dir = [] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.102804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.102888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] command line args: [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 10 13:59:16.102987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] config_source = [] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.103075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 10 13:59:16.103153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] debug = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.103224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ================================================================================ {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 10 13:59:16.103352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.103616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] admin_token = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.103749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] config_dir = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.103831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.103898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.103960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.104069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] config_source = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.104177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.104286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] debug = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.104384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.104486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.104618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] list_limit = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.104718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.104825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] log_config_append = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.104928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.105024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.105125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.105226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] log_dir = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.105226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] log_file = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.105463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.105578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.105704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.105704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_config_append = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.105883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.105957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_dir = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_file = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.106922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.107069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.107152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.107229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.107343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.107420 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.107599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.107728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.107833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.107911 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.108076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.108194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.108317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.108433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.108549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.108672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.108779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.108995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.109112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.109239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] publish_errors = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.109353 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.109472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.109594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.109718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.109823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.109966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.110085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.110209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] publish_errors = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.110325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.110439 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.110551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] standard_threads = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.110685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.110813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.110929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.111050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.111165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.111286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.111409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.111566 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] use_journal = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.111566 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] use_json = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.111784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] standard_threads = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.111898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] use_stderr = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.112014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.112196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] use_syslog = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.112316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.112431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.112551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.112676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.112792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] use_journal = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.112792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] use_json = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.113017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.113132 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] use_stderr = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.113246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.113360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] use_syslog = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.113480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.113608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 10 13:59:16.113724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.113838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.113953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.114067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.114181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.114304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.114426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.114548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.114664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.114779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.114894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.115019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.115135 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.115256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.115372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.115525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.115642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.115769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.115902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.116040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.116208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.116378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.116505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.116635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.116758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.116875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.116875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.117090 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.117204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.117204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.117518 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.117638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.117764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.117881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.118073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.118190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.118306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.118421 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.118542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.118657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.118783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.118899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.119014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.119129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.119242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.119364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.119532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.119729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.119858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.119980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.120096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.120210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.120333 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.120451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.120565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.120685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.120807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.120923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.121046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.121163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.121278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.121392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.121508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.121628 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.121756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.121908 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.122026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.122141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.122255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.122369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.122492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.122612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.122727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.122841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.122962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.123077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.123200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.123316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.123430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.123562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.123676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.123790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.123985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.124103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.124217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.124332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.124446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.124559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.124687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.124809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.124925 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.125039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.125153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.125267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.125389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.125505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.125660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.125775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.125775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.125775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.125775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.126159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.126262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.126359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.126462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.126565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.126671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.126769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.126871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.127004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.127132 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.127258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.127374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.127551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.127670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.127792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.127909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.128029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.128145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.128261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.128382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.128507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.128637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.128765 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.128883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.129011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.129130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.129252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.129374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.129491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.129606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.129720 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.129842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.129958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.130074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.130195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.130301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.130434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.130559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.130676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.130798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.130905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.131019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.131142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.131260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.131375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.132054 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.132208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.132311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.132421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.132520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.132630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.132747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.132862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.132978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.133108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.133227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.133227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.133227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.133227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.133634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.133758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.133882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.133998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.134115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.134271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.134390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.134514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.134632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.134747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.134863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.134984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.135102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.135227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.135344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.135486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.135486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.135767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.135885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.136081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.136202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.136317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.136432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.136546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.136670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.136799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.136925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.137042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.137157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.137274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.137389 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.137504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.137628 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.137745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.137861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.137976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.138091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.138207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.138332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.138449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.138565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.138679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.138679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.138901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.138901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.139140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.139263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.external = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.139380 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.139510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.139657 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.139657 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.139898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.139898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.140132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.140249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.password = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.140366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] auth.external = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.140482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] auth.token = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.140598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.140726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.140844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.140962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.141078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.141195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.141311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] auth.password = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.141436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.141554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] auth.token = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.141670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.141787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.141904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.142020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.142145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.142263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.142380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.142497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.142617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.142734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.142859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.142977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.143094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.143210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.143326 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.143483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] credential.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.143617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.143736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.143852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.143968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.144085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.144208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.144337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.144455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.144572 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.144689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.144805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.144921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.144921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.145149 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.145266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.145382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.145499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.145616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.145779 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.145900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.146023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.146149 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.146268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.146385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.146385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.146614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.146731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.146853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.146970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.147198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.147833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.147936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] federation.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.148032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.148230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.148311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.148393 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.148467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.148539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.148610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.148717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.148853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.148972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.149993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.150064 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.150180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.150298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.150415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.150540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.150659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.150775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.150893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.151010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.151127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.151251 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.151376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.151544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.151663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.151780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.151897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.152030 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.152150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.152267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.152384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.152500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.152615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.152744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.152863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.152981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.153110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.153228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.153345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.153470 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.153588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.153705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.153821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.153938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.154055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.154180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.154298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.154421 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.154539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.154656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.154772 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.154896 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.155016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.155133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.155250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.155366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.155495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.155626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.155745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.155901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.156018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.156136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.156253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.156378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.156502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.156620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.156736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.156853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.156977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.157095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.157212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.157329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.157446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.157562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.157687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.157805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.158027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.158151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.158269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.158386 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.158512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.158631 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.158748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.158865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.158982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.158982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.159209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.159328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.159475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.159595 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.159712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.159828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.159957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.159957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.160178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.160296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.160412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.160529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.160654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.160772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.160889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.161005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.161121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.161246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.161374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.161492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.161608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.161735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.161853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.161970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.162094 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.162212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.162329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.162447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.162562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.162720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.162853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.163156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.163278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.163397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.163527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.163644 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.163774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.163894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.164011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.164127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.164244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.164360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.164486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.164604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.164720 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.164837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.164837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.165062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.165188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.165307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.165423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.165423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.165643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.165760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.165921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.166042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.166158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.166275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.166392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.166509 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.166634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.166753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.166870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.166986 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.167103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.167219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.167344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.167519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.167642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.167759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.167875 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.167992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.168117 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.168236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.168352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.168469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.168586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.168703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.168828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.168947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.169063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.169180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.169296 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.169413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.169538 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.169656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.169774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.169891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.170008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.170124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.170249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.170368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.170485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.170602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.170718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.170835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.170960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.171079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.171195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.171195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.171414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.171547 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.171676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.171797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] resource.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.171914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.172031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.172147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.172147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.172376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.172493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] role.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.172608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.172724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] role.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.172841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.173038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.173158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.173275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.173391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.173508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.173625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.173750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.173868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.173868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.174088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.174205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] role.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.174322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.174447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] role.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.174566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.174683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.174799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.174915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.175032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.175157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.175276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.175392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.175563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.175681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.175799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.175970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.176093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.176210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.176327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.176443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.176559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.176684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.176803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.176920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.177038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.177155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.177271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.177396 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.177515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.177632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.177749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.177865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.177982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.178107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.178226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.178378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.178548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.178714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.178875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.178996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.179114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.179232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.179349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.179484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.179618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.179738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.179855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.179972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.180089 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.180207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.180332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.180451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.180568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.180685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.180802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.180918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.181043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.181161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.181279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.181279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.181498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.181621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.181621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.181851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.181968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.182084 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.182200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.182317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.182442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.182560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.182676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.182793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.182913 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.183031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] token.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.183155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.183274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.183392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.183525 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.183643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.183759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.183888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.184007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.184123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.184240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.184356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.184472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.184596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.184779 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.184909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.185028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.185144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.185262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.185384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.connection = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.185512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.185631 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.185748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.185865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.185982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.186138 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.186264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.186382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.186500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.186617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.186733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.186858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.186984 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.187103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.187228 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.187347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.187493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.connection = **** {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.187615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.187732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.187849 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.187978 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.188097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.188214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.188330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.188447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.188564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.188690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.188808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.188924 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.189041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.189156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.189273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.189398 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.189516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.189634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.189750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.189865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.189982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81589) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 10 13:59:16.190108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.190108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.190108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.190108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.190108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.190108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.190108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.190108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 10 13:59:16.190108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81588) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 10 13:59:16.216787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.217014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.217237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.217994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.218480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.218702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.225847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.227298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.253395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.253621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.254828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.255058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.307535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.308962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.356725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.356951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.357531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.357761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.513196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.513471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.513609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.513826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.514041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.514172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.514303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.514446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.514572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.514707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.514905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.515118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.537739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.537973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.538060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.538200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.538419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.538503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.538638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.538718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.538853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.538937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.539066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.539146 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.539279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.539520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.550114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.550275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.563260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.563469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.563683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.563794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.563902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.563985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.564116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.564198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.564328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.564409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.564538 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.564619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81589) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.564746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.564953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81588) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 10 13:59:16.581420 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.581627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 13:59:16.581710 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.581910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 13:59:16.581990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.582150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 13:59:16.582231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.582424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.582505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 10 13:59:16.582685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.582774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.582942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.583022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.583196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.583274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.583467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.593873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.594147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.594238 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.594415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.594501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.594693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.594781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.595023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.601749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.602309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.604667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.604951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.605201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.605285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.605482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.605564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.605760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.605843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.606018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.606100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.606287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.606379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.606553 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.606635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.606817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.606898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.607086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.607170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.607352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.607432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.607645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.607913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.630684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.631108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.632735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.633016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.633434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.633718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.637758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.638043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.638323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.638403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.638623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.638899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.644753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.645038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.645214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.645498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.650725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.651022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.651312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.651472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.651621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.651759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.651896 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.652047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.652168 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.652334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.652425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.652612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.652693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.652885 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.653150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.653414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.668225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.668504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.669225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.669497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.672325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.673306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.675530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.675829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.676514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.676805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.680775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.681781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.683965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.684865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.687169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.687486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.687777 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.688063 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.688183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.688354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.688477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.688651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.688757 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.689039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.689325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.689602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.705295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.705583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.705849 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.706108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.706190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.706420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.706687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.706949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.713261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.714218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.714803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.715768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.716347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.716620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.717287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.717569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.719594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.719877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.720158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.720449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.720528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.720754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.720836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.721034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.721315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.721696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.732493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.732779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.733061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.733341 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.733602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.733722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.733886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.734000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.734167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.734277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.734446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.734831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.735109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.735392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.750137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.751660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.753060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.754550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.755966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.757456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.757775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.758911 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.759193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.759273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.759619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.760677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.760954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.761352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.765915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.767362 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.767652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.769098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.770261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.770658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.770945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.771223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.771981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.772673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.772860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.773139 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.779582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.779876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.780157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.780913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.781211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.781488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.787008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81589) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.787291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.787595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.788512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.788788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.789060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.789150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81588) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 10 13:59:16.789339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.789423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.789615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.789696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.789879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.789970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.790159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.790250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.790429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.790519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.790687 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.790769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.790956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.791037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.791219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81589) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.791343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.791532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.791806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.792071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.792335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.792607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81588) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 10 13:59:16.825861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e73ba1c9c0 pid: 81589 (default app) Sep 10 13:59:16.827489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55e73ba1c9c0 pid: 81588 (default app) Sep 10 13:59:16.829096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821889df-1033-40ec-82a6-95586ef76d8d None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:16.829382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821889df-1033-40ec-82a6-95586ef76d8d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:16.829650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821889df-1033-40ec-82a6-95586ef76d8d None None] PATH_INFO: `/v3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:16.833778 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Sat Sep 10 13:59:16 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:23.649667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc4956-1f86-4d02-9c24-c62e99690942 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:23.650176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc4956-1f86-4d02-9c24-c62e99690942 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:23.650282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffc4956-1f86-4d02-9c24-c62e99690942 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:23.654479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 13:59:23.659837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:23.660310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:23.660399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:23.731209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: INFO dbcounter [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] Registered counter for database keystone Sep 10 13:59:23.750523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG oslo_db.sqlalchemy.engines [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81589) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 10 13:59:23.757341 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writer thread running {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 10 13:59:23.935704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG passlib.handlers.bcrypt [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81589) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 10 13:59:23.936081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG passlib.handlers.bcrypt [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81589) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 10 13:59:23.994276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:23.995825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:24.125552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 10 13:59:24.427487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc4956-1f86-4d02-9c24-c62e99690942 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:24.498624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: INFO dbcounter [None req-cffc4956-1f86-4d02-9c24-c62e99690942 None None] Registered counter for database keystone Sep 10 13:59:24.520226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG oslo_db.sqlalchemy.engines [None req-cffc4956-1f86-4d02-9c24-c62e99690942 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81588) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 10 13:59:24.526890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writer thread running {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 10 13:59:24.645853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffc4956-1f86-4d02-9c24-c62e99690942 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:24.650758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e17efaa-3bc4-428b-b784-c4b88c46bb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:24.651384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17efaa-3bc4-428b-b784-c4b88c46bb79 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:24.651694 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17efaa-3bc4-428b-b784-c4b88c46bb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:24.651970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e17efaa-3bc4-428b-b784-c4b88c46bb79 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:24.660146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17efaa-3bc4-428b-b784-c4b88c46bb79 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:24.719496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0e17efaa-3bc4-428b-b784-c4b88c46bb79 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 10 13:59:24.719785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e17efaa-3bc4-428b-b784-c4b88c46bb79 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:24.730145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Sat Sep 10 13:59:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 13:59:24.736132 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:24.753423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b344a2-77a2-45af-9b76-b8bd200a441a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:24.758320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298641a-dddb-48d6-a6df-b85b2f4472ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:24.759416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298641a-dddb-48d6-a6df-b85b2f4472ce admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:24.759720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298641a-dddb-48d6-a6df-b85b2f4472ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:24.759987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298641a-dddb-48d6-a6df-b85b2f4472ce admin admin] PATH_INFO: `/v3/projects/7a00398c85824c23b37ffe5fee90b840` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:24.761547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298641a-dddb-48d6-a6df-b85b2f4472ce admin admin] RBAC: Authorizing `identity:get_project(project_id=7a00398c85824c23b37ffe5fee90b840)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:24.815145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8298641a-dddb-48d6-a6df-b85b2f4472ce admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 10 13:59:24.815501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8298641a-dddb-48d6-a6df-b85b2f4472ce admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:24.820252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Sat Sep 10 13:59:24 2022] GET /identity/v3/projects/7a00398c85824c23b37ffe5fee90b840 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:26.939418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6abc1-915a-42ed-94a4-242ca0221d14 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:26.939967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6abc1-915a-42ed-94a4-242ca0221d14 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:26.940064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f6abc1-915a-42ed-94a4-242ca0221d14 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:26.944091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:26.949742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c189872-6edf-4185-9522-33c8806527da None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:26.950167 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c189872-6edf-4185-9522-33c8806527da None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:26.950289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c189872-6edf-4185-9522-33c8806527da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:26.970834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7c189872-6edf-4185-9522-33c8806527da None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:26.972364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7c189872-6edf-4185-9522-33c8806527da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:26.980816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:26 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 10 13:59:27.253456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6abc1-915a-42ed-94a4-242ca0221d14 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:27.266306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f6abc1-915a-42ed-94a4-242ca0221d14 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:27.270567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d187b9-9e4f-41b9-9086-04c29f947bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:27.271167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d187b9-9e4f-41b9-9086-04c29f947bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:27.271470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d187b9-9e4f-41b9-9086-04c29f947bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:27.271751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d187b9-9e4f-41b9-9086-04c29f947bf3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:27.279612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d187b9-9e4f-41b9-9086-04c29f947bf3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:27.281387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d187b9-9e4f-41b9-9086-04c29f947bf3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:27.290632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Sat Sep 10 13:59:27 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 13:59:27.295965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c189872-6edf-4185-9522-33c8806527da None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:27.305574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c189872-6edf-4185-9522-33c8806527da None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:27.309977 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd5cb1-8b58-4495-8f0b-b921a9c0c441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:27.310594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd5cb1-8b58-4495-8f0b-b921a9c0c441 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:27.310872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd5cb1-8b58-4495-8f0b-b921a9c0c441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:27.311144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd5cb1-8b58-4495-8f0b-b921a9c0c441 admin admin] PATH_INFO: `/v3/users/25a17c1158c94c229e16c77c070bd9d7` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:27.312476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd5cb1-8b58-4495-8f0b-b921a9c0c441 admin admin] RBAC: Authorizing `identity:get_user(user_id=25a17c1158c94c229e16c77c070bd9d7)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:27.314329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd5cb1-8b58-4495-8f0b-b921a9c0c441 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:27.315881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Sat Sep 10 13:59:27 2022] GET /identity/v3/users/25a17c1158c94c229e16c77c070bd9d7 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:29.536539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a9e0e-4037-4889-b79d-c59658785f8f None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:29.537067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a9e0e-4037-4889-b79d-c59658785f8f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:29.537067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a9e0e-4037-4889-b79d-c59658785f8f None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:29.539073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:29.544336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c167b5d-9f88-4be8-bc8c-9c3fec6f9923 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:29.544767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c167b5d-9f88-4be8-bc8c-9c3fec6f9923 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:29.544998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c167b5d-9f88-4be8-bc8c-9c3fec6f9923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:29.562711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890629f8-013d-45ce-958b-7f42287d547b None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:29.562992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890629f8-013d-45ce-958b-7f42287d547b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:29.563272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890629f8-013d-45ce-958b-7f42287d547b None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:29.563361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5c167b5d-9f88-4be8-bc8c-9c3fec6f9923 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:29.564810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5c167b5d-9f88-4be8-bc8c-9c3fec6f9923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:29.564925 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 13:59:29.570155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef5c68-f054-4ad7-8e01-afcc82b16d3e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:29.570431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef5c68-f054-4ad7-8e01-afcc82b16d3e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:29.570705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef5c68-f054-4ad7-8e01-afcc82b16d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:29.612553 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG passlib.handlers.bcrypt [None req-39ef5c68-f054-4ad7-8e01-afcc82b16d3e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81588) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 10 13:59:29.612930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG passlib.handlers.bcrypt [None req-39ef5c68-f054-4ad7-8e01-afcc82b16d3e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81588) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 10 13:59:29.646035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:29.662244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-39ef5c68-f054-4ad7-8e01-afcc82b16d3e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:29.663983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-39ef5c68-f054-4ad7-8e01-afcc82b16d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:29.762566 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:29.914831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c167b5d-9f88-4be8-bc8c-9c3fec6f9923 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:29.939236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:30.021001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c167b5d-9f88-4be8-bc8c-9c3fec6f9923 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:30.030211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef5c68-f054-4ad7-8e01-afcc82b16d3e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:30.055191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f6fbe-113e-4ec1-a37c-60b542f6df4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:30.055853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f6fbe-113e-4ec1-a37c-60b542f6df4d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:30.056130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f6fbe-113e-4ec1-a37c-60b542f6df4d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:30.056400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f6fbe-113e-4ec1-a37c-60b542f6df4d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:30.067289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f6fbe-113e-4ec1-a37c-60b542f6df4d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-854f6fbe-113e-4ec1-a37c-60b542f6df4d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:30.076158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:30.077159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 13:59:30.078037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 13:59:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 13:59:30.082956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f6fbe-113e-4ec1-a37c-60b542f6df4d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:30.130534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef5c68-f054-4ad7-8e01-afcc82b16d3e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:30.133068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:30.164770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:30.170107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a2869-7260-43b8-b8aa-ef990b80fd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:30.170971 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2869-7260-43b8-b8aa-ef990b80fd1e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:30.171704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2869-7260-43b8-b8aa-ef990b80fd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:30.171985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a2869-7260-43b8-b8aa-ef990b80fd1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:30.183196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f6fbe-113e-4ec1-a37c-60b542f6df4d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:30.216791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaae19e-3723-4736-9a9e-56967c9dd105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:30.217382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaae19e-3723-4736-9a9e-56967c9dd105 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:30.217657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaae19e-3723-4736-9a9e-56967c9dd105 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:30.217923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaae19e-3723-4736-9a9e-56967c9dd105 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:30.218838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaae19e-3723-4736-9a9e-56967c9dd105 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:30.219130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaae19e-3723-4736-9a9e-56967c9dd105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:30.220560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaae19e-3723-4736-9a9e-56967c9dd105 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:30.232719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 13:59:30.238312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaae19e-3723-4736-9a9e-56967c9dd105 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:30.239464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a2869-7260-43b8-b8aa-ef990b80fd1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:30.241173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a2869-7260-43b8-b8aa-ef990b80fd1e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:30.336391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaae19e-3723-4736-9a9e-56967c9dd105 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:30.338742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 13:59:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 13:59:30.344582 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a2869-7260-43b8-b8aa-ef990b80fd1e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:30.348045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:30.373428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af8f09-4b58-4450-8334-e3923bab53ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:30.374119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af8f09-4b58-4450-8334-e3923bab53ad None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:30.374399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af8f09-4b58-4450-8334-e3923bab53ad None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:30.374674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af8f09-4b58-4450-8334-e3923bab53ad None admin] PATH_INFO: `/v3/users/25a17c1158c94c229e16c77c070bd9d7` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:30.375835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af8f09-4b58-4450-8334-e3923bab53ad None admin] RBAC: Authorizing `identity:get_user(user_id=25a17c1158c94c229e16c77c070bd9d7)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:30.377292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af8f09-4b58-4450-8334-e3923bab53ad None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:30.378860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:30 2022] GET /identity/v3/users/25a17c1158c94c229e16c77c070bd9d7 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 13:59:30.385068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af8f09-4b58-4450-8334-e3923bab53ad None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:30.399563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:30.448239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a2869-7260-43b8-b8aa-ef990b80fd1e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:30.482147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088ccfa-d5e6-4510-a20d-06e423213cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:30.482996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088ccfa-d5e6-4510-a20d-06e423213cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:30.483277 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088ccfa-d5e6-4510-a20d-06e423213cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:30.483578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088ccfa-d5e6-4510-a20d-06e423213cf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:30.485295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af8f09-4b58-4450-8334-e3923bab53ad None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:30.502776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088ccfa-d5e6-4510-a20d-06e423213cf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:30.504373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e088ccfa-d5e6-4510-a20d-06e423213cf6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:30.520296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14528751-d23c-4e8b-862c-ed8e75c21afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:30.520915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14528751-d23c-4e8b-862c-ed8e75c21afd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:30.521188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14528751-d23c-4e8b-862c-ed8e75c21afd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:30.521455 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14528751-d23c-4e8b-862c-ed8e75c21afd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-e088ccfa-d5e6-4510-a20d-06e423213cf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 13:59:30.522459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:30.523486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 13:59:30.524320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 13:59:30.524320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 13:59:30.524320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 13:59:30.524481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14528751-d23c-4e8b-862c-ed8e75c21afd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:30.524545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 13:59:30.524615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14528751-d23c-4e8b-862c-ed8e75c21afd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:30.530084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 13:59:30.530217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088ccfa-d5e6-4510-a20d-06e423213cf6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:30.530970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14528751-d23c-4e8b-862c-ed8e75c21afd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:30.587963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:30.612780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:30.631413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e088ccfa-d5e6-4510-a20d-06e423213cf6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:30.633337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14528751-d23c-4e8b-862c-ed8e75c21afd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:30.664360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d0b1d-e050-4286-9c77-7d54fe072fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:30.664973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d0b1d-e050-4286-9c77-7d54fe072fdc None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:30.665255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d0b1d-e050-4286-9c77-7d54fe072fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:30.665523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d0b1d-e050-4286-9c77-7d54fe072fdc None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:30.666176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d0b1d-e050-4286-9c77-7d54fe072fdc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:30.666515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d0b1d-e050-4286-9c77-7d54fe072fdc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:30.667098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4607ae82-6046-4fb9-9e6a-12a8de8f1bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:30.667734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607ae82-6046-4fb9-9e6a-12a8de8f1bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:30.667907 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d0b1d-e050-4286-9c77-7d54fe072fdc None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:30.668015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607ae82-6046-4fb9-9e6a-12a8de8f1bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:30.668287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4607ae82-6046-4fb9-9e6a-12a8de8f1bd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:30.669008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607ae82-6046-4fb9-9e6a-12a8de8f1bd0 None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.domain.id=default, user.id=25a17c1158c94c229e16c77c070bd9d7) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:30.669314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607ae82-6046-4fb9-9e6a-12a8de8f1bd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:30.670737 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4607ae82-6046-4fb9-9e6a-12a8de8f1bd0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:30.677237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Sat Sep 10 13:59:30 2022] GET /identity/v3/role_assignments?user.id=25a17c1158c94c229e16c77c070bd9d7&scope.domain.id=default&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:30.685711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 13:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:33.460553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909f697-afec-4ce5-a602-6a53b16697a9 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:33.461504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909f697-afec-4ce5-a602-6a53b16697a9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:33.462006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2909f697-afec-4ce5-a602-6a53b16697a9 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:33.463557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:33.469193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bee7a-293c-4810-9af9-c35c73cc3301 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:33.469659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bee7a-293c-4810-9af9-c35c73cc3301 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:33.469736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6bee7a-293c-4810-9af9-c35c73cc3301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:33.493050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2c6bee7a-293c-4810-9af9-c35c73cc3301 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:33.494592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2c6bee7a-293c-4810-9af9-c35c73cc3301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:33.516504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63221d-4ed5-4c5c-8ada-82471b730f99 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:33.516855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63221d-4ed5-4c5c-8ada-82471b730f99 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:33.517046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a63221d-4ed5-4c5c-8ada-82471b730f99 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:33.518616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 13/23] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:33.523822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd65a02-9400-44bb-9103-2416e48bb323 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:33.524104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd65a02-9400-44bb-9103-2416e48bb323 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:33.524379 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd65a02-9400-44bb-9103-2416e48bb323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:33.560481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7dd65a02-9400-44bb-9103-2416e48bb323 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:33.561697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7dd65a02-9400-44bb-9103-2416e48bb323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:33.591091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 11/24] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:33.692874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 14/25] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:33.768335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb25e9-2505-430a-a791-3601e6e423ac None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:33.768695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb25e9-2505-430a-a791-3601e6e423ac None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:33.768887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb25e9-2505-430a-a791-3601e6e423ac None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:33.770456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 12/26] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:33.784834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8a87-1495-43b2-bb02-c167be1409a7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:33.785119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8a87-1495-43b2-bb02-c167be1409a7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:33.785396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa8a87-1495-43b2-bb02-c167be1409a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:33.812434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e3fa8a87-1495-43b2-bb02-c167be1409a7 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:33.813621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e3fa8a87-1495-43b2-bb02-c167be1409a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:33.828941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d47a6-cc0c-45ad-8ff6-315709838685 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:33.829229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d47a6-cc0c-45ad-8ff6-315709838685 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:33.829481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d47a6-cc0c-45ad-8ff6-315709838685 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:33.830997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 13/27] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 13:59:33.840394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1f4ed-90b9-4651-8752-a3f2c26f3d2b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:33.840673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1f4ed-90b9-4651-8752-a3f2c26f3d2b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:33.841045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1f4ed-90b9-4651-8752-a3f2c26f3d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:33.876703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-09d1f4ed-90b9-4651-8752-a3f2c26f3d2b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:33.878106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-09d1f4ed-90b9-4651-8752-a3f2c26f3d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:33.934154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 15/28] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:33.967870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:33.999818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 14/29] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:34.065634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa8a87-1495-43b2-bb02-c167be1409a7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:34.070945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:34.169334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1f4ed-90b9-4651-8752-a3f2c26f3d2b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:34.193535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa8a87-1495-43b2-bb02-c167be1409a7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:34.263248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b92ca-eaf0-4cc1-9861-325fd6778f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:34.263924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b92ca-eaf0-4cc1-9861-325fd6778f9c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:34.264263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b92ca-eaf0-4cc1-9861-325fd6778f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:34.264599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529b92ca-eaf0-4cc1-9861-325fd6778f9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:34.282699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b92ca-eaf0-4cc1-9861-325fd6778f9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:34.284436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529b92ca-eaf0-4cc1-9861-325fd6778f9c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-529b92ca-eaf0-4cc1-9861-325fd6778f9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 13:59:34.308070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:34.308981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 13:59:34.309751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 13:59:34.309751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 13:59:34.309751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 13:59:34.309751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 16/30] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:34.315386 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b92ca-eaf0-4cc1-9861-325fd6778f9c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:34.387056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1f4ed-90b9-4651-8752-a3f2c26f3d2b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:34.402222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:34.453247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529b92ca-eaf0-4cc1-9861-325fd6778f9c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:34.479872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002528f4-621f-4bd5-a52d-5eff3f3f7afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:34.481241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002528f4-621f-4bd5-a52d-5eff3f3f7afc None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:34.481727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002528f4-621f-4bd5-a52d-5eff3f3f7afc None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:34.482202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002528f4-621f-4bd5-a52d-5eff3f3f7afc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:34.491927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59975723-0834-408f-a55d-7f64874ff48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:34.492542 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59975723-0834-408f-a55d-7f64874ff48d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:34.492816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59975723-0834-408f-a55d-7f64874ff48d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:34.493088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59975723-0834-408f-a55d-7f64874ff48d None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:34.493738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59975723-0834-408f-a55d-7f64874ff48d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:34.494048 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59975723-0834-408f-a55d-7f64874ff48d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:34.495460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59975723-0834-408f-a55d-7f64874ff48d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:34.497777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002528f4-621f-4bd5-a52d-5eff3f3f7afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-002528f4-621f-4bd5-a52d-5eff3f3f7afc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:34.504961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:34.505774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:34.507373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 13:59:34.512191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 15/31] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 13:59:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 13:59:34.512191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002528f4-621f-4bd5-a52d-5eff3f3f7afc None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:34.520157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 17/32] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 13:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:34.521142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59975723-0834-408f-a55d-7f64874ff48d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:34.555269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:34.625718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002528f4-621f-4bd5-a52d-5eff3f3f7afc None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:34.636040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59975723-0834-408f-a55d-7f64874ff48d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:34.655330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:34.663168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad8a85-e1f8-429d-a01d-3800697dc398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:34.663890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad8a85-e1f8-429d-a01d-3800697dc398 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:34.664263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad8a85-e1f8-429d-a01d-3800697dc398 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:34.664628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ad8a85-e1f8-429d-a01d-3800697dc398 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:34.665419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad8a85-e1f8-429d-a01d-3800697dc398 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:34.666914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ad8a85-e1f8-429d-a01d-3800697dc398 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:34.680190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170273f-9f7a-4626-a04a-8522a39ccf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:34.681151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170273f-9f7a-4626-a04a-8522a39ccf6e None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:34.681781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170273f-9f7a-4626-a04a-8522a39ccf6e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:34.682240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170273f-9f7a-4626-a04a-8522a39ccf6e None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:34.683202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170273f-9f7a-4626-a04a-8522a39ccf6e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:34.685003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b170273f-9f7a-4626-a04a-8522a39ccf6e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:34.803215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 16/33] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Sep 10 13:59:34 2022] POST /identity/v3/roles => generated 225 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 13:59:34.804572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad8a85-e1f8-429d-a01d-3800697dc398 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:34.818291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 18/34] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Sep 10 13:59:34 2022] POST /identity/v3/roles => generated 219 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 13:59:34.820498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170273f-9f7a-4626-a04a-8522a39ccf6e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:34.940658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b170273f-9f7a-4626-a04a-8522a39ccf6e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:34.989831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd637881-4f57-4436-91ab-48eda6c7625e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:34.992058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd637881-4f57-4436-91ab-48eda6c7625e None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:34.993440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd637881-4f57-4436-91ab-48eda6c7625e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:34.994764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd637881-4f57-4436-91ab-48eda6c7625e None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:34.995782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd637881-4f57-4436-91ab-48eda6c7625e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:34.997904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd637881-4f57-4436-91ab-48eda6c7625e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:35.032675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:35.046787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ad8a85-e1f8-429d-a01d-3800697dc398 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:35.059575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 19/35] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 13:59:34 2022] POST /identity/v3/projects => generated 301 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 13:59:35.072795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf0d05-11e6-4682-a0ce-10a55f3d618b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:35.073112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf0d05-11e6-4682-a0ce-10a55f3d618b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:35.073481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf0d05-11e6-4682-a0ce-10a55f3d618b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:35.075013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 20/36] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 13:59:35.081134 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e43043-01cd-48b1-bdf7-de51a47ab873 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:35.081451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e43043-01cd-48b1-bdf7-de51a47ab873 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:35.081691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e43043-01cd-48b1-bdf7-de51a47ab873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:35.104988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11a62e-5765-4a3d-a843-2596c060d35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:35.106072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11a62e-5765-4a3d-a843-2596c060d35a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:35.106597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11a62e-5765-4a3d-a843-2596c060d35a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:35.107076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f11a62e-5765-4a3d-a843-2596c060d35a None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:35.108086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11a62e-5765-4a3d-a843-2596c060d35a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:35.108546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f2e43043-01cd-48b1-bdf7-de51a47ab873 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:35.108817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11a62e-5765-4a3d-a843-2596c060d35a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:35.110130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f2e43043-01cd-48b1-bdf7-de51a47ab873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:35.110910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f11a62e-5765-4a3d-a843-2596c060d35a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:35.143708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 17/37] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:35.150822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:35.160565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11a62e-5765-4a3d-a843-2596c060d35a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:35.212928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 21/38] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:35.281650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457127cf-4d9c-4220-86b7-2851d3961767 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:35.281964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457127cf-4d9c-4220-86b7-2851d3961767 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:35.282239 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457127cf-4d9c-4220-86b7-2851d3961767 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:35.283845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 22/39] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 13:59:35.289184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368812f-0695-4bc5-8924-cc29224adaac None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:35.289442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368812f-0695-4bc5-8924-cc29224adaac None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:35.289738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9368812f-0695-4bc5-8924-cc29224adaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:35.307036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f11a62e-5765-4a3d-a843-2596c060d35a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:35.315496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9368812f-0695-4bc5-8924-cc29224adaac None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:35.317299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9368812f-0695-4bc5-8924-cc29224adaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:35.345100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45f6c7-020e-41aa-a816-7ae8be7e5f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:35.346203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45f6c7-020e-41aa-a816-7ae8be7e5f94 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:35.346695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45f6c7-020e-41aa-a816-7ae8be7e5f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:35.347184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45f6c7-020e-41aa-a816-7ae8be7e5f94 None admin] PATH_INFO: `/v3/users/25a17c1158c94c229e16c77c070bd9d7` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:35.351115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45f6c7-020e-41aa-a816-7ae8be7e5f94 None admin] RBAC: Authorizing `identity:get_user(user_id=25a17c1158c94c229e16c77c070bd9d7)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:35.352895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45f6c7-020e-41aa-a816-7ae8be7e5f94 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:35.355188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 18/40] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:35 2022] GET /identity/v3/users/25a17c1158c94c229e16c77c070bd9d7 => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:35.362502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45f6c7-020e-41aa-a816-7ae8be7e5f94 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:35.405609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:35.508324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45f6c7-020e-41aa-a816-7ae8be7e5f94 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:35.520183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 23/41] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:35.537074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368812f-0695-4bc5-8924-cc29224adaac None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:35.543721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c00aee-92f9-47ed-8926-5d6eeda4d790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:35.544372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c00aee-92f9-47ed-8926-5d6eeda4d790 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:35.544654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c00aee-92f9-47ed-8926-5d6eeda4d790 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:35.544923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c00aee-92f9-47ed-8926-5d6eeda4d790 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:35.546138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c00aee-92f9-47ed-8926-5d6eeda4d790 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:35.547627 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c00aee-92f9-47ed-8926-5d6eeda4d790 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:35.549204 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 19/42] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:35.555385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c00aee-92f9-47ed-8926-5d6eeda4d790 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:35.632040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:35.644979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9368812f-0695-4bc5-8924-cc29224adaac None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:35.654368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:35.664493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c00aee-92f9-47ed-8926-5d6eeda4d790 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:35.682695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80855fd8-4ccf-427a-866f-c80ac9388c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:35.683636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80855fd8-4ccf-427a-866f-c80ac9388c1b None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:35.684115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80855fd8-4ccf-427a-866f-c80ac9388c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:35.684584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80855fd8-4ccf-427a-866f-c80ac9388c1b None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:35.685595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80855fd8-4ccf-427a-866f-c80ac9388c1b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:35.687169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80855fd8-4ccf-427a-866f-c80ac9388c1b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:35.701880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bacacd-ab79-45ae-8494-66b866a32126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:35.702596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bacacd-ab79-45ae-8494-66b866a32126 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:35.702817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bacacd-ab79-45ae-8494-66b866a32126 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:35.703088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bacacd-ab79-45ae-8494-66b866a32126 None admin] PATH_INFO: `/v3/domains/default/users/25a17c1158c94c229e16c77c070bd9d7/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:35.707480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 24/43] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Sep 10 13:59:35 2022] POST /identity/v3/roles => generated 223 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 13:59:35.707639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bacacd-ab79-45ae-8494-66b866a32126 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=25a17c1158c94c229e16c77c070bd9d7, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:35.708185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bacacd-ab79-45ae-8494-66b866a32126 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:35.724501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 20/44] 10.222.0.26 () {58 vars in 1361 bytes} [Sat Sep 10 13:59:35 2022] PUT /identity/v3/domains/default/users/25a17c1158c94c229e16c77c070bd9d7/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 13:59:35.795514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80855fd8-4ccf-427a-866f-c80ac9388c1b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:35.921711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80855fd8-4ccf-427a-866f-c80ac9388c1b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:35.959616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78e1336-9cd1-4347-aebf-7b76d671fe96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:35.960263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e1336-9cd1-4347-aebf-7b76d671fe96 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:35.960551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e1336-9cd1-4347-aebf-7b76d671fe96 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:35.960831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78e1336-9cd1-4347-aebf-7b76d671fe96 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:35.962011 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e1336-9cd1-4347-aebf-7b76d671fe96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:35.963542 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78e1336-9cd1-4347-aebf-7b76d671fe96 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:35.965290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 25/45] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:35.970947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bacacd-ab79-45ae-8494-66b866a32126 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:36.076813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bacacd-ab79-45ae-8494-66b866a32126 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:36.110011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0eb5e-fc76-4c8d-a2c0-3f7247916e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:36.110628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0eb5e-fc76-4c8d-a2c0-3f7247916e3c None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:36.110913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0eb5e-fc76-4c8d-a2c0-3f7247916e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:36.111189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0eb5e-fc76-4c8d-a2c0-3f7247916e3c None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:36.111891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0eb5e-fc76-4c8d-a2c0-3f7247916e3c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:36.113354 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe0eb5e-fc76-4c8d-a2c0-3f7247916e3c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:36.143499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 21/46] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 13:59:35 2022] POST /identity/v3/projects => generated 312 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 13:59:37.941545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ef3e-08df-416c-a50d-b39ea14f0391 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:37.942059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ef3e-08df-416c-a50d-b39ea14f0391 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:37.942249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ef3e-08df-416c-a50d-b39ea14f0391 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:37.943664 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 26/47] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:37.949374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110c90c-ccf6-490e-aa1d-a0079396adf5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:37.949790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110c90c-ccf6-490e-aa1d-a0079396adf5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:37.949926 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e110c90c-ccf6-490e-aa1d-a0079396adf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:37.962784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:37.968862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e110c90c-ccf6-490e-aa1d-a0079396adf5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:37.970712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e110c90c-ccf6-490e-aa1d-a0079396adf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:38.044289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 22/48] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:38.303742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7485-4890-4fe0-b836-4414e7f89ce0 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:38.304108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7485-4890-4fe0-b836-4414e7f89ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:38.304246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722a7485-4890-4fe0-b836-4414e7f89ce0 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:38.305601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 27/49] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:38.310911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37854c-d148-43d8-ab6a-cfe604196c07 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:38.311264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37854c-d148-43d8-ab6a-cfe604196c07 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:38.311641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf37854c-d148-43d8-ab6a-cfe604196c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:38.313580 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a7485-4890-4fe0-b836-4414e7f89ce0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:38.330781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-bf37854c-d148-43d8-ab6a-cfe604196c07 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:38.331989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bf37854c-d148-43d8-ab6a-cfe604196c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:38.332790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:38.422018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722a7485-4890-4fe0-b836-4414e7f89ce0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:38.461898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e8e1b-8640-4e1f-ae62-aabcc1a69eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:38.462547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e8e1b-8640-4e1f-ae62-aabcc1a69eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:38.462824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e8e1b-8640-4e1f-ae62-aabcc1a69eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:38.463093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e8e1b-8640-4e1f-ae62-aabcc1a69eb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:38.475266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e8e1b-8640-4e1f-ae62-aabcc1a69eb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-251e8e1b-8640-4e1f-ae62-aabcc1a69eb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:38.480600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:38.481418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 13:59:38.482156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 28/50] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 13:59:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 13:59:38.487653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e8e1b-8640-4e1f-ae62-aabcc1a69eb1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:38.509505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 23/51] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:38.564504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:38.586859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e8e1b-8640-4e1f-ae62-aabcc1a69eb1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:38.620100 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79997170-c36f-446c-bc6e-aa065b634e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:38.620698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79997170-c36f-446c-bc6e-aa065b634e1f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:38.620979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79997170-c36f-446c-bc6e-aa065b634e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:38.621252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79997170-c36f-446c-bc6e-aa065b634e1f None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:38.621906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79997170-c36f-446c-bc6e-aa065b634e1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:38.622209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79997170-c36f-446c-bc6e-aa065b634e1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:38.623637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79997170-c36f-446c-bc6e-aa065b634e1f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:38.632116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 29/52] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 13:59:38.637820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37854c-d148-43d8-ab6a-cfe604196c07 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:38.701345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:38.736290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf37854c-d148-43d8-ab6a-cfe604196c07 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:38.771847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080e94b-8e51-4f9f-b2cb-339d4f874578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:38.772496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080e94b-8e51-4f9f-b2cb-339d4f874578 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:38.772777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080e94b-8e51-4f9f-b2cb-339d4f874578 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:38.773052 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6080e94b-8e51-4f9f-b2cb-339d4f874578 None admin] PATH_INFO: `/v3/users/25a17c1158c94c229e16c77c070bd9d7` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:38.774177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080e94b-8e51-4f9f-b2cb-339d4f874578 None admin] RBAC: Authorizing `identity:get_user(user_id=25a17c1158c94c229e16c77c070bd9d7)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:38.775656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6080e94b-8e51-4f9f-b2cb-339d4f874578 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:38.777392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 24/53] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:38 2022] GET /identity/v3/users/25a17c1158c94c229e16c77c070bd9d7 => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:38.777977 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79997170-c36f-446c-bc6e-aa065b634e1f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:38.783423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080e94b-8e51-4f9f-b2cb-339d4f874578 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:38.881199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79997170-c36f-446c-bc6e-aa065b634e1f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:38.884448 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6080e94b-8e51-4f9f-b2cb-339d4f874578 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:38.916593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca3a13-d29a-44a4-8e49-63093350bbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:38.917218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca3a13-d29a-44a4-8e49-63093350bbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:38.917496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca3a13-d29a-44a4-8e49-63093350bbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:38.917769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ca3a13-d29a-44a4-8e49-63093350bbb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:38.918275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e39dd7-48c3-4bf9-bebd-4e78fe3db2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:38.918862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca3a13-d29a-44a4-8e49-63093350bbb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:38.918949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e39dd7-48c3-4bf9-bebd-4e78fe3db2f4 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:38.919173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e39dd7-48c3-4bf9-bebd-4e78fe3db2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:38.919470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e39dd7-48c3-4bf9-bebd-4e78fe3db2f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:38.920365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ca3a13-d29a-44a4-8e49-63093350bbb1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:38.920594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e39dd7-48c3-4bf9-bebd-4e78fe3db2f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:38.921917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 30/54] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 13:59:38.922043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e39dd7-48c3-4bf9-bebd-4e78fe3db2f4 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:38.925307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 25/55] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 13:59:38.927640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca3a13-d29a-44a4-8e49-63093350bbb1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:38.929884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e39dd7-48c3-4bf9-bebd-4e78fe3db2f4 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:38.932465 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:38.937119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:39.030448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ca3a13-d29a-44a4-8e49-63093350bbb1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:39.032459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e39dd7-48c3-4bf9-bebd-4e78fe3db2f4 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:39.064239 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f883650-1da5-48da-88ec-cf7fc03bcb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:39.064843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883650-1da5-48da-88ec-cf7fc03bcb25 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:39.065122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883650-1da5-48da-88ec-cf7fc03bcb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:39.065400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f883650-1da5-48da-88ec-cf7fc03bcb25 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:39.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f883650-1da5-48da-88ec-cf7fc03bcb25 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:39.067189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57df8733-ae5a-4f52-a367-ba5bcd0a45a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:39.067516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f883650-1da5-48da-88ec-cf7fc03bcb25 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:39.067830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df8733-ae5a-4f52-a367-ba5bcd0a45a0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:39.068108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df8733-ae5a-4f52-a367-ba5bcd0a45a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:39.068384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57df8733-ae5a-4f52-a367-ba5bcd0a45a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:39.069879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df8733-ae5a-4f52-a367-ba5bcd0a45a0 None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.domain.id=default, user.id=25a17c1158c94c229e16c77c070bd9d7) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:39.070189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df8733-ae5a-4f52-a367-ba5bcd0a45a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:39.071649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57df8733-ae5a-4f52-a367-ba5bcd0a45a0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:39.079395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 26/56] 10.222.0.26 () {56 vars in 1322 bytes} [Sat Sep 10 13:59:38 2022] GET /identity/v3/role_assignments?user.id=25a17c1158c94c229e16c77c070bd9d7&scope.domain.id=default&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:39.095574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 31/57] 10.222.0.26 () {60 vars in 1140 bytes} [Sat Sep 10 13:59:38 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 13:59:41.270953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68836acd-fcde-4a6b-8c42-1907b6543083 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:41.271502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68836acd-fcde-4a6b-8c42-1907b6543083 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:41.271588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68836acd-fcde-4a6b-8c42-1907b6543083 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:41.273384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 27/58] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:41.278528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282940f2-a96d-4172-8a4e-b0e56df8791d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:41.278957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282940f2-a96d-4172-8a4e-b0e56df8791d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:41.279079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282940f2-a96d-4172-8a4e-b0e56df8791d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:41.297504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-282940f2-a96d-4172-8a4e-b0e56df8791d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:41.298699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-282940f2-a96d-4172-8a4e-b0e56df8791d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:41.350588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:41.365110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 32/59] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:41.632737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68836acd-fcde-4a6b-8c42-1907b6543083 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:41.697579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:41.733074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68836acd-fcde-4a6b-8c42-1907b6543083 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:41.766966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4186ade3-3171-497f-990d-febf489ee4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:41.767623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186ade3-3171-497f-990d-febf489ee4ee None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:41.767902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186ade3-3171-497f-990d-febf489ee4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:41.768173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4186ade3-3171-497f-990d-febf489ee4ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:41.769281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186ade3-3171-497f-990d-febf489ee4ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:41.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4186ade3-3171-497f-990d-febf489ee4ee None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:41.772569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 28/60] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:41.778068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282940f2-a96d-4172-8a4e-b0e56df8791d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:41.875367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282940f2-a96d-4172-8a4e-b0e56df8791d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:41.908272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1a03d9-4345-4cc0-9dec-f26a4025e039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:41.908817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a03d9-4345-4cc0-9dec-f26a4025e039 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:41.909098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a03d9-4345-4cc0-9dec-f26a4025e039 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:41.909372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1a03d9-4345-4cc0-9dec-f26a4025e039 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:41.910015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a03d9-4345-4cc0-9dec-f26a4025e039 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:41.911479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1a03d9-4345-4cc0-9dec-f26a4025e039 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:41.938213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 33/61] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 13:59:41 2022] POST /identity/v3/users => generated 275 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 13:59:45.304901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0080931-a5b0-40f5-919a-5de6bca1bb11 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:45.305833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0080931-a5b0-40f5-919a-5de6bca1bb11 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:45.306317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0080931-a5b0-40f5-919a-5de6bca1bb11 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:45.307958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 29/62] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:45.313909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50f5cc-dc5d-4c1d-bac4-105123c4df8f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:45.314419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50f5cc-dc5d-4c1d-bac4-105123c4df8f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:45.314490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa50f5cc-dc5d-4c1d-bac4-105123c4df8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:45.340846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-aa50f5cc-dc5d-4c1d-bac4-105123c4df8f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:45.342085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-aa50f5cc-dc5d-4c1d-bac4-105123c4df8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:45.342470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c0b35-2180-4807-9ee6-59f7ea761d7a None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:45.342946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c0b35-2180-4807-9ee6-59f7ea761d7a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:45.343402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c0b35-2180-4807-9ee6-59f7ea761d7a None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:45.345247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 30/63] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:45.356694 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c1469-6519-4acb-af1f-f1ef360739b8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:45.357292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c1469-6519-4acb-af1f-f1ef360739b8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:45.357563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c1469-6519-4acb-af1f-f1ef360739b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:45.388501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-eb1c1469-6519-4acb-af1f-f1ef360739b8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:45.390662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-eb1c1469-6519-4acb-af1f-f1ef360739b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:45.417353 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:45.461371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 34/64] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:45.462811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bb167-bf86-4af3-88fc-888471199ae9 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:45.463268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bb167-bf86-4af3-88fc-888471199ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:45.463746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bb167-bf86-4af3-88fc-888471199ae9 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:45.465152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 35/65] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:45.471097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db33ce-c4b5-4f69-b26b-be00672b9892 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:45.471376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db33ce-c4b5-4f69-b26b-be00672b9892 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:45.471674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45db33ce-c4b5-4f69-b26b-be00672b9892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:45.506866 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-45db33ce-c4b5-4f69-b26b-be00672b9892 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:45.508244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-45db33ce-c4b5-4f69-b26b-be00672b9892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:45.517069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 31/66] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:45.585724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 36/67] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:45.766937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c1469-6519-4acb-af1f-f1ef360739b8 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:45.823042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb3f4c-2858-4a3d-89f2-804501f4a1ac None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:45.823385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb3f4c-2858-4a3d-89f2-804501f4a1ac None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:45.823685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcb3f4c-2858-4a3d-89f2-804501f4a1ac None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:45.824989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 37/68] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 13:59:45.832514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21071f-c00e-47e2-89ba-de3d4119eb30 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:45.832801 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21071f-c00e-47e2-89ba-de3d4119eb30 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:45.833083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db21071f-c00e-47e2-89ba-de3d4119eb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:45.846734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:45.867086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-db21071f-c00e-47e2-89ba-de3d4119eb30 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:45.868623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-db21071f-c00e-47e2-89ba-de3d4119eb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:45.906326 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c1469-6519-4acb-af1f-f1ef360739b8 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:45.940964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 38/69] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 10 13:59:45.942183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21071f-c00e-47e2-89ba-de3d4119eb30 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:45.949434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a745-8eae-4d18-ad42-0720f69bbddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:45.950112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a745-8eae-4d18-ad42-0720f69bbddb None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:45.950396 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a745-8eae-4d18-ad42-0720f69bbddb None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:45.950671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a6a745-8eae-4d18-ad42-0720f69bbddb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:45.963969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a6a745-8eae-4d18-ad42-0720f69bbddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-d4a6a745-8eae-4d18-ad42-0720f69bbddb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:45.969889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:45.970805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 13:59:45.971586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 13:59:45.974418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 32/70] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:45.976491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a745-8eae-4d18-ad42-0720f69bbddb None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:45.990175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:46.069492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db21071f-c00e-47e2-89ba-de3d4119eb30 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.095470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a6a745-8eae-4d18-ad42-0720f69bbddb None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.110679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbcea8-c7dc-4a55-ab75-efd001a2d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.111356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbcea8-c7dc-4a55-ab75-efd001a2d967 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.111658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbcea8-c7dc-4a55-ab75-efd001a2d967 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.111929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbcea8-c7dc-4a55-ab75-efd001a2d967 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.120997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:46.125129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbcea8-c7dc-4a55-ab75-efd001a2d967 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9ffbcea8-c7dc-4a55-ab75-efd001a2d967 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:46.130607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:46.131834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 39/71] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 13:59:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:46.132585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbcea8-c7dc-4a55-ab75-efd001a2d967 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:46.139830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4656055-86a4-4c15-94cc-d7ef8e3a59ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.140481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4656055-86a4-4c15-94cc-d7ef8e3a59ca None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.140763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4656055-86a4-4c15-94cc-d7ef8e3a59ca None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.141040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4656055-86a4-4c15-94cc-d7ef8e3a59ca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.152306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4656055-86a4-4c15-94cc-d7ef8e3a59ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-f4656055-86a4-4c15-94cc-d7ef8e3a59ca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:46.157507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:46.158340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 13:59:46.159064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 33/72] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 13:59:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:46.160447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac79a72-a516-49d9-9f89-ba291eef1c21 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.160447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac79a72-a516-49d9-9f89-ba291eef1c21 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.160930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac79a72-a516-49d9-9f89-ba291eef1c21 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.162714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 34/73] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:46.164840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac79a72-a516-49d9-9f89-ba291eef1c21 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:46.242555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:46.249695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbcea8-c7dc-4a55-ab75-efd001a2d967 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.272480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac79a72-a516-49d9-9f89-ba291eef1c21 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.286701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc0d3e-3828-45f1-ba4d-e00869cccb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.287466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0d3e-3828-45f1-ba4d-e00869cccb0c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.287753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0d3e-3828-45f1-ba4d-e00869cccb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.288028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bc0d3e-3828-45f1-ba4d-e00869cccb0c None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.288698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc0d3e-3828-45f1-ba4d-e00869cccb0c None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:46.289002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc0d3e-3828-45f1-ba4d-e00869cccb0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:46.290410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bc0d3e-3828-45f1-ba4d-e00869cccb0c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:46.298399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 40/74] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 13:59:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:46.299261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc0d3e-3828-45f1-ba4d-e00869cccb0c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:46.305671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54ba3f-6e1d-4087-a4da-27a7af55acd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.306280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54ba3f-6e1d-4087-a4da-27a7af55acd8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.306556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54ba3f-6e1d-4087-a4da-27a7af55acd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.306830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54ba3f-6e1d-4087-a4da-27a7af55acd8 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.307551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54ba3f-6e1d-4087-a4da-27a7af55acd8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:46.307856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54ba3f-6e1d-4087-a4da-27a7af55acd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:46.309363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54ba3f-6e1d-4087-a4da-27a7af55acd8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:46.317890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 35/75] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:46.320881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3893b-6a2d-4ab7-8f3b-b1c0de88a035 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.321176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3893b-6a2d-4ab7-8f3b-b1c0de88a035 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.321457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a3893b-6a2d-4ab7-8f3b-b1c0de88a035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.340768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-82a3893b-6a2d-4ab7-8f3b-b1c0de88a035 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:46.342386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-82a3893b-6a2d-4ab7-8f3b-b1c0de88a035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:46.383649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:46.400900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bc0d3e-3828-45f1-ba4d-e00869cccb0c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.409562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 36/76] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:46.410400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3893b-6a2d-4ab7-8f3b-b1c0de88a035 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:46.448964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecc8d2-badb-4e37-ba6b-8fc23761f40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.449575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecc8d2-badb-4e37-ba6b-8fc23761f40d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.449848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecc8d2-badb-4e37-ba6b-8fc23761f40d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.450123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecc8d2-badb-4e37-ba6b-8fc23761f40d None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.450795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecc8d2-badb-4e37-ba6b-8fc23761f40d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:46.451096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecc8d2-badb-4e37-ba6b-8fc23761f40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:46.452509 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecc8d2-badb-4e37-ba6b-8fc23761f40d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:46.461864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 41/77] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 13:59:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:46.462797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecc8d2-badb-4e37-ba6b-8fc23761f40d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:46.485298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:46.533862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a3893b-6a2d-4ab7-8f3b-b1c0de88a035 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.570612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0f68e-a6db-41c0-8a01-7e4eb5468101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.571221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecc8d2-badb-4e37-ba6b-8fc23761f40d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.571327 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0f68e-a6db-41c0-8a01-7e4eb5468101 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.571535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0f68e-a6db-41c0-8a01-7e4eb5468101 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.571806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0f68e-a6db-41c0-8a01-7e4eb5468101 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.582944 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0f68e-a6db-41c0-8a01-7e4eb5468101 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-45b0f68e-a6db-41c0-8a01-7e4eb5468101 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:46.588932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:46.589861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 37/78] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:46.592219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0f68e-a6db-41c0-8a01-7e4eb5468101 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:46.609472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71174d64-9f7f-4d4d-a376-5dd9f179348a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.610112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71174d64-9f7f-4d4d-a376-5dd9f179348a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.610389 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71174d64-9f7f-4d4d-a376-5dd9f179348a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.610661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71174d64-9f7f-4d4d-a376-5dd9f179348a None admin] PATH_INFO: `/v3/users/a072f8abd5f2437cbc748eae7f1f4f83` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.623928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71174d64-9f7f-4d4d-a376-5dd9f179348a None admin] RBAC: Authorizing `identity:get_user(user_id=a072f8abd5f2437cbc748eae7f1f4f83)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:46.625422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71174d64-9f7f-4d4d-a376-5dd9f179348a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:46.631474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 42/79] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:46 2022] GET /identity/v3/users/a072f8abd5f2437cbc748eae7f1f4f83 => generated 275 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:46.631474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71174d64-9f7f-4d4d-a376-5dd9f179348a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:46.637986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:46.706027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0f68e-a6db-41c0-8a01-7e4eb5468101 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.706729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:46.731274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71174d64-9f7f-4d4d-a376-5dd9f179348a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.741134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e65126-c252-44f4-9495-3154b556a972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.741767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e65126-c252-44f4-9495-3154b556a972 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.742046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e65126-c252-44f4-9495-3154b556a972 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.742319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e65126-c252-44f4-9495-3154b556a972 None admin] PATH_INFO: `/v3/users/25a17c1158c94c229e16c77c070bd9d7` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.743480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e65126-c252-44f4-9495-3154b556a972 None admin] RBAC: Authorizing `identity:get_user(user_id=25a17c1158c94c229e16c77c070bd9d7)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:46.745302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e65126-c252-44f4-9495-3154b556a972 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:46.746767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 38/80] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:46 2022] GET /identity/v3/users/25a17c1158c94c229e16c77c070bd9d7 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:46.747338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e65126-c252-44f4-9495-3154b556a972 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:46.770158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0d15d-938b-41c9-b732-2c0604724ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.770795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0d15d-938b-41c9-b732-2c0604724ff8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.771075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0d15d-938b-41c9-b732-2c0604724ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.771367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0d15d-938b-41c9-b732-2c0604724ff8 None admin] PATH_INFO: `/v3/users/a072f8abd5f2437cbc748eae7f1f4f83` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.772605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0d15d-938b-41c9-b732-2c0604724ff8 None admin] RBAC: Authorizing `identity:get_user(user_id=a072f8abd5f2437cbc748eae7f1f4f83)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:46.774068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0d15d-938b-41c9-b732-2c0604724ff8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:46.775694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 43/81] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:46 2022] GET /identity/v3/users/a072f8abd5f2437cbc748eae7f1f4f83 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:46.776454 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0d15d-938b-41c9-b732-2c0604724ff8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:46.851890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e65126-c252-44f4-9495-3154b556a972 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.854267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:46.880371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0d15d-938b-41c9-b732-2c0604724ff8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:46.886281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879b2e6-fc4d-49c4-b187-3bdf972361cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.886889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879b2e6-fc4d-49c4-b187-3bdf972361cf None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.887167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879b2e6-fc4d-49c4-b187-3bdf972361cf None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.887456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0879b2e6-fc4d-49c4-b187-3bdf972361cf None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.888179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879b2e6-fc4d-49c4-b187-3bdf972361cf None admin] RBAC: Adding query filter params (name=member) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:46.888437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879b2e6-fc4d-49c4-b187-3bdf972361cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:46.889830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0879b2e6-fc4d-49c4-b187-3bdf972361cf None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:46.897844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 39/82] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 13:59:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:46.897844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879b2e6-fc4d-49c4-b187-3bdf972361cf None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:46.913973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d66ec-c6b6-44aa-b892-222421738a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:46.914580 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d66ec-c6b6-44aa-b892-222421738a63 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:46.914863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d66ec-c6b6-44aa-b892-222421738a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:46.915132 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d66ec-c6b6-44aa-b892-222421738a63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:46.929835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:46.983108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d66ec-c6b6-44aa-b892-222421738a63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:46.984619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6d66ec-c6b6-44aa-b892-222421738a63 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:47.000841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0879b2e6-fc4d-49c4-b187-3bdf972361cf None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:47.035906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e380f-ae5f-45ae-a1dc-ef41df4f01ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:47.036531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e380f-ae5f-45ae-a1dc-ef41df4f01ef None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:47.036806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e380f-ae5f-45ae-a1dc-ef41df4f01ef None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:47.037154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e380f-ae5f-45ae-a1dc-ef41df4f01ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:47.038452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e380f-ae5f-45ae-a1dc-ef41df4f01ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:47.039957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e380f-ae5f-45ae-a1dc-ef41df4f01ef None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:47.041503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 40/83] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:47.042247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e380f-ae5f-45ae-a1dc-ef41df4f01ef None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:47.077702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:47.085012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 44/84] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 13:59:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 13:59:47.085872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d66ec-c6b6-44aa-b892-222421738a63 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:47.130433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:47.149188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e380f-ae5f-45ae-a1dc-ef41df4f01ef None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:47.182835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654e6f4-d3fe-45ef-bff0-9677308f6441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:47.183472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654e6f4-d3fe-45ef-bff0-9677308f6441 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:47.183754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654e6f4-d3fe-45ef-bff0-9677308f6441 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:47.184032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f654e6f4-d3fe-45ef-bff0-9677308f6441 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:47.184567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d66ec-c6b6-44aa-b892-222421738a63 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:47.218016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0df0b-7936-4034-a9e9-d8f84a60b521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:47.218880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0df0b-7936-4034-a9e9-d8f84a60b521 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:47.219159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0df0b-7936-4034-a9e9-d8f84a60b521 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:47.219432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd0df0b-7936-4034-a9e9-d8f84a60b521 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:47.249959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654e6f4-d3fe-45ef-bff0-9677308f6441 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:47.251436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f654e6f4-d3fe-45ef-bff0-9677308f6441 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:47.284822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0df0b-7936-4034-a9e9-d8f84a60b521 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:47.286056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:47.286314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd0df0b-7936-4034-a9e9-d8f84a60b521 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:47.330039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:47.351719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 41/85] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 13:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 13:59:47.352571 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654e6f4-d3fe-45ef-bff0-9677308f6441 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:47.388512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 45/86] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 13:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 13:59:47.389238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0df0b-7936-4034-a9e9-d8f84a60b521 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:47.452697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f654e6f4-d3fe-45ef-bff0-9677308f6441 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:47.487279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71baf56-e1cc-4132-9d1f-900bea47725d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:47.487520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd0df0b-7936-4034-a9e9-d8f84a60b521 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:47.487946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71baf56-e1cc-4132-9d1f-900bea47725d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:47.488199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71baf56-e1cc-4132-9d1f-900bea47725d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:47.488470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71baf56-e1cc-4132-9d1f-900bea47725d None admin] PATH_INFO: `/v3/users/a072f8abd5f2437cbc748eae7f1f4f83` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:47.489592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71baf56-e1cc-4132-9d1f-900bea47725d None admin] RBAC: Authorizing `identity:get_user(user_id=a072f8abd5f2437cbc748eae7f1f4f83)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:47.491074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71baf56-e1cc-4132-9d1f-900bea47725d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:47.492674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 42/87] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:47 2022] GET /identity/v3/users/a072f8abd5f2437cbc748eae7f1f4f83 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:47.493428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71baf56-e1cc-4132-9d1f-900bea47725d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:47.495941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:47.524681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaab147-22f6-467b-9ffb-a9f9c8b0f5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:47.525292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaab147-22f6-467b-9ffb-a9f9c8b0f5b7 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:47.525574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaab147-22f6-467b-9ffb-a9f9c8b0f5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:47.525852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaab147-22f6-467b-9ffb-a9f9c8b0f5b7 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:47.526507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaab147-22f6-467b-9ffb-a9f9c8b0f5b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:47.528018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaab147-22f6-467b-9ffb-a9f9c8b0f5b7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:47.545667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:47.550389 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 46/88] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 13:59:47 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 13:59:47.551178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaab147-22f6-467b-9ffb-a9f9c8b0f5b7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:47.619743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71baf56-e1cc-4132-9d1f-900bea47725d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:47.656512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f97ed-d341-4e01-a59a-faf65d8db17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:47.657161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f97ed-d341-4e01-a59a-faf65d8db17e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:47.657441 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f97ed-d341-4e01-a59a-faf65d8db17e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:47.657715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216f97ed-d341-4e01-a59a-faf65d8db17e None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:47.658904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f97ed-d341-4e01-a59a-faf65d8db17e None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:47.660423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216f97ed-d341-4e01-a59a-faf65d8db17e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:47.662690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 43/89] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 13:59:47 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:47.664454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f97ed-d341-4e01-a59a-faf65d8db17e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:47.671413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaab147-22f6-467b-9ffb-a9f9c8b0f5b7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:47.710261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894e1de-e2f9-4a1f-aee4-7b5a732b24d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:47.710840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894e1de-e2f9-4a1f-aee4-7b5a732b24d4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:47.711123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894e1de-e2f9-4a1f-aee4-7b5a732b24d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:47.711400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894e1de-e2f9-4a1f-aee4-7b5a732b24d4 None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:47.712649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894e1de-e2f9-4a1f-aee4-7b5a732b24d4 None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:47.714639 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894e1de-e2f9-4a1f-aee4-7b5a732b24d4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:47.717060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 47/90] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 13:59:47 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:47.717852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894e1de-e2f9-4a1f-aee4-7b5a732b24d4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:47.749442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:47.786069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216f97ed-d341-4e01-a59a-faf65d8db17e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:47.796560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:47.828205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2d22f-d64a-4713-9834-95e7d6633fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:47.829120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2d22f-d64a-4713-9834-95e7d6633fdd None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:47.829456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2d22f-d64a-4713-9834-95e7d6633fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:47.829744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba2d22f-d64a-4713-9834-95e7d6633fdd None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:47.831011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2d22f-d64a-4713-9834-95e7d6633fdd None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:47.832574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba2d22f-d64a-4713-9834-95e7d6633fdd None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:47.834355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 44/91] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 13:59:47 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:47.835222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2d22f-d64a-4713-9834-95e7d6633fdd None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:47.842180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894e1de-e2f9-4a1f-aee4-7b5a732b24d4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:47.887367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9ad4d-8d3a-4585-a5a4-1a44a466358a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:47.888115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9ad4d-8d3a-4585-a5a4-1a44a466358a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:47.888398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9ad4d-8d3a-4585-a5a4-1a44a466358a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:47.888672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada9ad4d-8d3a-4585-a5a4-1a44a466358a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:47.950418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba2d22f-d64a-4713-9834-95e7d6633fdd None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:47.975545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9ad4d-8d3a-4585-a5a4-1a44a466358a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:47.977338 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada9ad4d-8d3a-4585-a5a4-1a44a466358a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:47.990028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc509e-655c-45c4-a699-6a6607e844ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:47.991031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc509e-655c-45c4-a699-6a6607e844ab None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:47.991318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc509e-655c-45c4-a699-6a6607e844ab None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:47.991626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc509e-655c-45c4-a699-6a6607e844ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:47.992420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc509e-655c-45c4-a699-6a6607e844ab None admin] RBAC: Adding query filter params (role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808, user.id=a072f8abd5f2437cbc748eae7f1f4f83) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:47.992733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc509e-655c-45c4-a699-6a6607e844ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:47.994393 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc509e-655c-45c4-a699-6a6607e844ab None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:48.007559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 45/92] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 13:59:47 2022] GET /identity/v3/role_assignments?user.id=a072f8abd5f2437cbc748eae7f1f4f83&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:48.009596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc509e-655c-45c4-a699-6a6607e844ab None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:48.013404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:48.086080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:48.170424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 48/93] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 13:59:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 13:59:48.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9ad4d-8d3a-4585-a5a4-1a44a466358a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:48.191380 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc509e-655c-45c4-a699-6a6607e844ab None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:48.233362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69a1c4-fd10-4bce-b5a1-cad32a34402f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:48.234304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69a1c4-fd10-4bce-b5a1-cad32a34402f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:48.234789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69a1c4-fd10-4bce-b5a1-cad32a34402f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:48.235284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c69a1c4-fd10-4bce-b5a1-cad32a34402f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:48.236249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69a1c4-fd10-4bce-b5a1-cad32a34402f None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808, user.id=25a17c1158c94c229e16c77c070bd9d7) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:48.236783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69a1c4-fd10-4bce-b5a1-cad32a34402f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:48.238883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c69a1c4-fd10-4bce-b5a1-cad32a34402f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:48.245201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 46/94] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 13:59:48 2022] GET /identity/v3/role_assignments?user.id=25a17c1158c94c229e16c77c070bd9d7&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:48.246730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69a1c4-fd10-4bce-b5a1-cad32a34402f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:48.332199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:48.394686 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c69a1c4-fd10-4bce-b5a1-cad32a34402f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:48.408512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada9ad4d-8d3a-4585-a5a4-1a44a466358a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:48.466472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862f711-8d76-40cc-94cc-0fbe42c772c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:48.467132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862f711-8d76-40cc-94cc-0fbe42c772c6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:48.467419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862f711-8d76-40cc-94cc-0fbe42c772c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:48.470006 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f862f711-8d76-40cc-94cc-0fbe42c772c6 None admin] PATH_INFO: `/v3/projects/12ccf7fa23154beeb220ad57780c582e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:48.472021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862f711-8d76-40cc-94cc-0fbe42c772c6 None admin] RBAC: Authorizing `identity:get_project(project_id=12ccf7fa23154beeb220ad57780c582e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:48.476729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f862f711-8d76-40cc-94cc-0fbe42c772c6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:48.479795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5a99f6-eca5-4e7d-8e80-7a0cf7a70af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:48.480422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a99f6-eca5-4e7d-8e80-7a0cf7a70af3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:48.480702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a99f6-eca5-4e7d-8e80-7a0cf7a70af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:48.480983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5a99f6-eca5-4e7d-8e80-7a0cf7a70af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:48.481732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a99f6-eca5-4e7d-8e80-7a0cf7a70af3 None admin] RBAC: Adding query filter params (role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808, user.id=a072f8abd5f2437cbc748eae7f1f4f83) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:48.482041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a99f6-eca5-4e7d-8e80-7a0cf7a70af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:48.483580 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5a99f6-eca5-4e7d-8e80-7a0cf7a70af3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:48.485931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 47/95] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 13:59:48 2022] GET /identity/v3/projects/12ccf7fa23154beeb220ad57780c582e => generated 312 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:48.490760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 49/96] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 13:59:48 2022] GET /identity/v3/role_assignments?user.id=a072f8abd5f2437cbc748eae7f1f4f83&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:48.494491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862f711-8d76-40cc-94cc-0fbe42c772c6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:48.672327 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f862f711-8d76-40cc-94cc-0fbe42c772c6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:48.723030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0acf7-7f0e-4501-a0b9-8190c2e0cb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:48.724027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0acf7-7f0e-4501-a0b9-8190c2e0cb5f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:48.724543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0acf7-7f0e-4501-a0b9-8190c2e0cb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:48.725049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0acf7-7f0e-4501-a0b9-8190c2e0cb5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:48.726037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0acf7-7f0e-4501-a0b9-8190c2e0cb5f None admin] RBAC: Adding query filter params (role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, scope.project.id=12ccf7fa23154beeb220ad57780c582e, user.id=a072f8abd5f2437cbc748eae7f1f4f83) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:48.726569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0acf7-7f0e-4501-a0b9-8190c2e0cb5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:48.728324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0acf7-7f0e-4501-a0b9-8190c2e0cb5f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:48.731775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:48.739732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 48/97] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 13:59:48 2022] GET /identity/v3/role_assignments?user.id=a072f8abd5f2437cbc748eae7f1f4f83&scope.project.id=12ccf7fa23154beeb220ad57780c582e&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:50.515679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dea49-2cb8-4b00-aa70-cd0a86d082f7 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:50.516631 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dea49-2cb8-4b00-aa70-cd0a86d082f7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:50.517117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48dea49-2cb8-4b00-aa70-cd0a86d082f7 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:50.518664 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 50/98] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:50.524613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572075b5-502d-47d8-b65c-794e575939d0 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:50.525516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572075b5-502d-47d8-b65c-794e575939d0 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:50.525989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572075b5-502d-47d8-b65c-794e575939d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:50.550964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-572075b5-502d-47d8-b65c-794e575939d0 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:50.552201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-572075b5-502d-47d8-b65c-794e575939d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:50.585112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546de43-0b0e-49c8-9c7c-53eb4d351d74 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:50.585751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546de43-0b0e-49c8-9c7c-53eb4d351d74 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:50.586223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3546de43-0b0e-49c8-9c7c-53eb4d351d74 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:50.587868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 51/99] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:50.608097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd166307-66dc-47db-9f11-c7fa2c3eec10 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:50.608701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd166307-66dc-47db-9f11-c7fa2c3eec10 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:50.609197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd166307-66dc-47db-9f11-c7fa2c3eec10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:50.644464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fd166307-66dc-47db-9f11-c7fa2c3eec10 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:50.646018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fd166307-66dc-47db-9f11-c7fa2c3eec10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:50.663394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 49/100] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:50.684115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:50.758557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 52/101] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:51.214198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572075b5-502d-47d8-b65c-794e575939d0 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:51.269169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218005aa-113f-49db-b1b8-1c61385eff2b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:51.269803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218005aa-113f-49db-b1b8-1c61385eff2b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:51.270284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218005aa-113f-49db-b1b8-1c61385eff2b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:51.272442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 53/102] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:51.281324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce64871-d71f-4fe0-abd6-a4d135a46056 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:51.281937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce64871-d71f-4fe0-abd6-a4d135a46056 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:51.282414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce64871-d71f-4fe0-abd6-a4d135a46056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:51.317405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2ce64871-d71f-4fe0-abd6-a4d135a46056 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:51.319629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2ce64871-d71f-4fe0-abd6-a4d135a46056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:51.384109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572075b5-502d-47d8-b65c-794e575939d0 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:51.427364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 54/103] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:51.431841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:51.432975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce64871-d71f-4fe0-abd6-a4d135a46056 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:51.442612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988ad69c-5dd4-4a39-b0d0-70ac889545cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:51.447488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad69c-5dd4-4a39-b0d0-70ac889545cf None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:51.447488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad69c-5dd4-4a39-b0d0-70ac889545cf None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:51.447488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988ad69c-5dd4-4a39-b0d0-70ac889545cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:51.450474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ad69c-5dd4-4a39-b0d0-70ac889545cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:51.452040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988ad69c-5dd4-4a39-b0d0-70ac889545cf None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:51.453644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 50/104] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:51.454776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24149f8f-c7fc-4f3e-8c98-28287a6e0968 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:51.455639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24149f8f-c7fc-4f3e-8c98-28287a6e0968 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:51.457768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24149f8f-c7fc-4f3e-8c98-28287a6e0968 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:51.465343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 51/105] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:51 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:51.466794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24149f8f-c7fc-4f3e-8c98-28287a6e0968 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:51.646184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24149f8f-c7fc-4f3e-8c98-28287a6e0968 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:51.649321 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:51.660024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce64871-d71f-4fe0-abd6-a4d135a46056 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:51.689977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4056d44-e0dc-412e-80ee-fbcbd0bc806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:51.690856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4056d44-e0dc-412e-80ee-fbcbd0bc806a None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:51.691278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4056d44-e0dc-412e-80ee-fbcbd0bc806a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:51.691976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4056d44-e0dc-412e-80ee-fbcbd0bc806a None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:51.693233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4056d44-e0dc-412e-80ee-fbcbd0bc806a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:51.695218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4056d44-e0dc-412e-80ee-fbcbd0bc806a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:51.700692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d58ca-58c5-4664-9f75-6e79b61cd343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:51.701495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d58ca-58c5-4664-9f75-6e79b61cd343 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:51.701774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d58ca-58c5-4664-9f75-6e79b61cd343 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:51.702043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d58ca-58c5-4664-9f75-6e79b61cd343 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:51.715149 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d58ca-58c5-4664-9f75-6e79b61cd343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-401d58ca-58c5-4664-9f75-6e79b61cd343 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:51.721171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:51.722150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 13:59:51.722891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 13:59:51.725547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 55/106] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:51.727754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ca010-203f-46ec-8092-232a21f50e14 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:51.728290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ca010-203f-46ec-8092-232a21f50e14 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:51.728680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 52/107] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 13:59:51 2022] POST /identity/v3/projects => generated 302 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 13:59:51.729042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ca010-203f-46ec-8092-232a21f50e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:51.730070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef8304-c9ef-4b5d-af6f-839c7680d394 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:51.730360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef8304-c9ef-4b5d-af6f-839c7680d394 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:51.730729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef8304-c9ef-4b5d-af6f-839c7680d394 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:51.732089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 53/108] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:51.735631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef8304-c9ef-4b5d-af6f-839c7680d394 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:51.755068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ec6ca010-203f-46ec-8092-232a21f50e14 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:51.757081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ec6ca010-203f-46ec-8092-232a21f50e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:51.799061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:51.844646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 56/109] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:51.846491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe3b01-c859-4ab6-95a2-2bdfc86021c6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:51.846979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe3b01-c859-4ab6-95a2-2bdfc86021c6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:51.847465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe3b01-c859-4ab6-95a2-2bdfc86021c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:51.868742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c5fe3b01-c859-4ab6-95a2-2bdfc86021c6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:51.870376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c5fe3b01-c859-4ab6-95a2-2bdfc86021c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:51.886880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef8304-c9ef-4b5d-af6f-839c7680d394 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:51.947875 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080fc2a-5922-4b8e-b0e5-d0b78946b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:51.948622 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080fc2a-5922-4b8e-b0e5-d0b78946b58b None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:51.948940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080fc2a-5922-4b8e-b0e5-d0b78946b58b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:51.949221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d080fc2a-5922-4b8e-b0e5-d0b78946b58b None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:51.949954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080fc2a-5922-4b8e-b0e5-d0b78946b58b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:51.950269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080fc2a-5922-4b8e-b0e5-d0b78946b58b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:51.952581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d080fc2a-5922-4b8e-b0e5-d0b78946b58b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:51.962693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 54/110] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 13:59:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:51.967708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080fc2a-5922-4b8e-b0e5-d0b78946b58b None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:51.985269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 57/111] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:51.991120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe3b01-c859-4ab6-95a2-2bdfc86021c6 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:52.014902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:52.148731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d080fc2a-5922-4b8e-b0e5-d0b78946b58b None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:52.167379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:52.181397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe3b01-c859-4ab6-95a2-2bdfc86021c6 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:52.209044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2bef3-a711-46f3-be25-dec06593e971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:52.209695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2bef3-a711-46f3-be25-dec06593e971 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:52.209977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2bef3-a711-46f3-be25-dec06593e971 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:52.210254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a2bef3-a711-46f3-be25-dec06593e971 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:52.223940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a2bef3-a711-46f3-be25-dec06593e971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:52.227616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d81f3b-d48e-448e-b746-0ab6018206fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:52.228265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d81f3b-d48e-448e-b746-0ab6018206fe None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:52.228545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d81f3b-d48e-448e-b746-0ab6018206fe None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:52.228823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d81f3b-d48e-448e-b746-0ab6018206fe None admin] PATH_INFO: `/v3/users/c6ddcc76426246a08038d84953728f4c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-89a2bef3-a711-46f3-be25-dec06593e971 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:52.229888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:52.230689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 13:59:52.231859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 55/112] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 13:59:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:52.232673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2bef3-a711-46f3-be25-dec06593e971 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:52.252626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d81f3b-d48e-448e-b746-0ab6018206fe None admin] RBAC: Authorizing `identity:get_user(user_id=c6ddcc76426246a08038d84953728f4c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:52.254043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d81f3b-d48e-448e-b746-0ab6018206fe None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:52.256438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 58/113] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:51 2022] GET /identity/v3/users/c6ddcc76426246a08038d84953728f4c => generated 289 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:52.257075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d81f3b-d48e-448e-b746-0ab6018206fe None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:52.340556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:52.353740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a2bef3-a711-46f3-be25-dec06593e971 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:52.375432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d81f3b-d48e-448e-b746-0ab6018206fe None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:52.390071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f48ad-41a6-413d-915b-abeb5a8a3efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:52.390756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f48ad-41a6-413d-915b-abeb5a8a3efd None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:52.390975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f48ad-41a6-413d-915b-abeb5a8a3efd None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:52.391240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f48ad-41a6-413d-915b-abeb5a8a3efd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:52.402546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8f48ad-41a6-413d-915b-abeb5a8a3efd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-dc8f48ad-41a6-413d-915b-abeb5a8a3efd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:52.407514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:52.408319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 13:59:52.409054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 56/114] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:52.410493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b9bbf-c757-4ed8-a167-fe3ab18c6aef None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:52.410493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b9bbf-c757-4ed8-a167-fe3ab18c6aef None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:52.410493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b9bbf-c757-4ed8-a167-fe3ab18c6aef None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:52.415809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 57/115] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:52.415809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b9bbf-c757-4ed8-a167-fe3ab18c6aef None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:52.418541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a77d0d-966a-4dc6-843b-9d7cec05219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:52.419162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a77d0d-966a-4dc6-843b-9d7cec05219f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:52.419468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a77d0d-966a-4dc6-843b-9d7cec05219f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:52.419765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a77d0d-966a-4dc6-843b-9d7cec05219f None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:52.420450 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a77d0d-966a-4dc6-843b-9d7cec05219f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:52.420752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a77d0d-966a-4dc6-843b-9d7cec05219f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:52.422151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a77d0d-966a-4dc6-843b-9d7cec05219f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:52.434089 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 59/116] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:52.434720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a77d0d-966a-4dc6-843b-9d7cec05219f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:52.454565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:52.536029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b9bbf-c757-4ed8-a167-fe3ab18c6aef None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:52.538468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a77d0d-966a-4dc6-843b-9d7cec05219f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:52.571252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:52.571570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e314a-4102-49b8-9d25-353fdec11814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:52.572126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e314a-4102-49b8-9d25-353fdec11814 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:52.572761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e314a-4102-49b8-9d25-353fdec11814 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:52.572761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e314a-4102-49b8-9d25-353fdec11814 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:52.577103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9ee29-e5b4-4be8-99a9-bc4220dd6d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:52.577735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9ee29-e5b4-4be8-99a9-bc4220dd6d37 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:52.578019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9ee29-e5b4-4be8-99a9-bc4220dd6d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:52.578294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9ee29-e5b4-4be8-99a9-bc4220dd6d37 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:52.589492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee9ee29-e5b4-4be8-99a9-bc4220dd6d37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6ee9ee29-e5b4-4be8-99a9-bc4220dd6d37 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:52.595068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:52.595945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 13:59:52.597107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 60/117] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 13:59:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:52.597913 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9ee29-e5b4-4be8-99a9-bc4220dd6d37 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:52.649549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e314a-4102-49b8-9d25-353fdec11814 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:52.651744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e314a-4102-49b8-9d25-353fdec11814 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:52.682818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:52.703901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee9ee29-e5b4-4be8-99a9-bc4220dd6d37 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:52.740477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e1f04-114a-4979-b876-cd783284179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:52.741119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e1f04-114a-4979-b876-cd783284179c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:52.741369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e1f04-114a-4979-b876-cd783284179c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:52.741647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e1f04-114a-4979-b876-cd783284179c None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:52.742317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e1f04-114a-4979-b876-cd783284179c None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:52.742622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e1f04-114a-4979-b876-cd783284179c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:52.744063 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5e1f04-114a-4979-b876-cd783284179c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:52.752454 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 61/118] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 13:59:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:52.758792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80e614-f3c8-4630-b563-b3b2512d8409 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:52.759275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80e614-f3c8-4630-b563-b3b2512d8409 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:52.759376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80e614-f3c8-4630-b563-b3b2512d8409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:52.767255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 58/119] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 13:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 13:59:52.767255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e314a-4102-49b8-9d25-353fdec11814 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:52.782820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-df80e614-f3c8-4630-b563-b3b2512d8409 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:52.784036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-df80e614-f3c8-4630-b563-b3b2512d8409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:52.839647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:52.861188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 62/120] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:52.872349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80e614-f3c8-4630-b563-b3b2512d8409 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:52.874975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e314a-4102-49b8-9d25-353fdec11814 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:52.908374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:52.916734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc40a3-9519-4dcd-80cc-7e09fa9ee21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:52.917375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc40a3-9519-4dcd-80cc-7e09fa9ee21f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:52.917659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc40a3-9519-4dcd-80cc-7e09fa9ee21f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:52.917937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc40a3-9519-4dcd-80cc-7e09fa9ee21f None admin] PATH_INFO: `/v3/users/25a17c1158c94c229e16c77c070bd9d7` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:52.919095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc40a3-9519-4dcd-80cc-7e09fa9ee21f None admin] RBAC: Authorizing `identity:get_user(user_id=25a17c1158c94c229e16c77c070bd9d7)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:52.921348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfc40a3-9519-4dcd-80cc-7e09fa9ee21f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:52.922780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 59/121] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:52 2022] GET /identity/v3/users/25a17c1158c94c229e16c77c070bd9d7 => generated 247 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:52.923666 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc40a3-9519-4dcd-80cc-7e09fa9ee21f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:52.993188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80e614-f3c8-4630-b563-b3b2512d8409 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.028838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583791f-f1b5-45e4-a58d-76bf31895f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.029796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583791f-f1b5-45e4-a58d-76bf31895f33 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.030324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583791f-f1b5-45e4-a58d-76bf31895f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.030785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f583791f-f1b5-45e4-a58d-76bf31895f33 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.031661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583791f-f1b5-45e4-a58d-76bf31895f33 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:53.032219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583791f-f1b5-45e4-a58d-76bf31895f33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:53.033848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f583791f-f1b5-45e4-a58d-76bf31895f33 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:53.043041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 63/122] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 13:59:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:53.044215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583791f-f1b5-45e4-a58d-76bf31895f33 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:53.050836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfc40a3-9519-4dcd-80cc-7e09fa9ee21f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.090116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92bd6a6-6d11-43c0-8f04-820c4c9fa13a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.090674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bd6a6-6d11-43c0-8f04-820c4c9fa13a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.090961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bd6a6-6d11-43c0-8f04-820c4c9fa13a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.091270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92bd6a6-6d11-43c0-8f04-820c4c9fa13a None admin] PATH_INFO: `/v3/users/a072f8abd5f2437cbc748eae7f1f4f83` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.092410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92bd6a6-6d11-43c0-8f04-820c4c9fa13a None admin] RBAC: Authorizing `identity:get_user(user_id=a072f8abd5f2437cbc748eae7f1f4f83)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:53.094417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92bd6a6-6d11-43c0-8f04-820c4c9fa13a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:53.095339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:53.099542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 60/123] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:52 2022] GET /identity/v3/users/a072f8abd5f2437cbc748eae7f1f4f83 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:53.100845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92bd6a6-6d11-43c0-8f04-820c4c9fa13a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:53.161364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f583791f-f1b5-45e4-a58d-76bf31895f33 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.181982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:53.200053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eed15-ee30-4691-aace-efe6d3cefa63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.201042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eed15-ee30-4691-aace-efe6d3cefa63 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.201489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eed15-ee30-4691-aace-efe6d3cefa63 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.201779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4eed15-ee30-4691-aace-efe6d3cefa63 None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.202900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eed15-ee30-4691-aace-efe6d3cefa63 None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:53.204402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4eed15-ee30-4691-aace-efe6d3cefa63 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:53.206052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 64/124] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 13:59:53 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:53.206810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eed15-ee30-4691-aace-efe6d3cefa63 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:53.227308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92bd6a6-6d11-43c0-8f04-820c4c9fa13a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.263276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3e5fd-93d8-4111-9940-4646efbaa55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.263934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3e5fd-93d8-4111-9940-4646efbaa55e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.264216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3e5fd-93d8-4111-9940-4646efbaa55e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.264493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f3e5fd-93d8-4111-9940-4646efbaa55e None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.265635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3e5fd-93d8-4111-9940-4646efbaa55e None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:53.267100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f3e5fd-93d8-4111-9940-4646efbaa55e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:53.268781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 61/125] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 13:59:53 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:53.269546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3e5fd-93d8-4111-9940-4646efbaa55e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:53.328188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4eed15-ee30-4691-aace-efe6d3cefa63 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.344122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:53.366750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c4b23-d63a-436f-9640-10203a25e9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.367385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c4b23-d63a-436f-9640-10203a25e9b3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.367773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c4b23-d63a-436f-9640-10203a25e9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.368057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c4b23-d63a-436f-9640-10203a25e9b3 None admin] PATH_INFO: `/v3/users/a072f8abd5f2437cbc748eae7f1f4f83` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.369173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c4b23-d63a-436f-9640-10203a25e9b3 None admin] RBAC: Authorizing `identity:get_user(user_id=a072f8abd5f2437cbc748eae7f1f4f83)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:53.370642 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c4b23-d63a-436f-9640-10203a25e9b3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:53.376796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 65/126] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:53 2022] GET /identity/v3/users/a072f8abd5f2437cbc748eae7f1f4f83 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:53.376796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c4b23-d63a-436f-9640-10203a25e9b3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:53.382099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f3e5fd-93d8-4111-9940-4646efbaa55e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.417292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f95861-f5dc-464c-80cf-c00c43d3067f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.417920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f95861-f5dc-464c-80cf-c00c43d3067f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.418197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f95861-f5dc-464c-80cf-c00c43d3067f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.418464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f95861-f5dc-464c-80cf-c00c43d3067f None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.419602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f95861-f5dc-464c-80cf-c00c43d3067f None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:53.421037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f95861-f5dc-464c-80cf-c00c43d3067f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:53.427248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 62/127] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 13:59:53 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:53.427248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f95861-f5dc-464c-80cf-c00c43d3067f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:53.430837 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:53.487415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c4b23-d63a-436f-9640-10203a25e9b3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.524384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df78f23-bf8f-4637-a6c0-fd95797768aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.525010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df78f23-bf8f-4637-a6c0-fd95797768aa None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.525294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df78f23-bf8f-4637-a6c0-fd95797768aa None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.525565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df78f23-bf8f-4637-a6c0-fd95797768aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.535811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df78f23-bf8f-4637-a6c0-fd95797768aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:53.541812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f95861-f5dc-464c-80cf-c00c43d3067f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2df78f23-bf8f-4637-a6c0-fd95797768aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:53.541946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:53.542759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 66/128] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:53.544383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df78f23-bf8f-4637-a6c0-fd95797768aa None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:53.578107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216d293-481d-49cd-a097-23a80aa23ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.578713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216d293-481d-49cd-a097-23a80aa23ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.578997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216d293-481d-49cd-a097-23a80aa23ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.579275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216d293-481d-49cd-a097-23a80aa23ef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.580038 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216d293-481d-49cd-a097-23a80aa23ef3 None admin] RBAC: Adding query filter params (role.id=efc3726869ca4763809656e2e7e1646a, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808, user.id=c6ddcc76426246a08038d84953728f4c) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:53.580349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216d293-481d-49cd-a097-23a80aa23ef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:53.581800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216d293-481d-49cd-a097-23a80aa23ef3 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:53.587979 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 63/129] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 13:59:53 2022] GET /identity/v3/role_assignments?user.id=c6ddcc76426246a08038d84953728f4c&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=efc3726869ca4763809656e2e7e1646a => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:53.590366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:53.601313 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216d293-481d-49cd-a097-23a80aa23ef3 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:53.660192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df78f23-bf8f-4637-a6c0-fd95797768aa None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.702638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:53.703163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ef77f-80fa-460c-83c7-f4d4cf74387d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.704131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ef77f-80fa-460c-83c7-f4d4cf74387d None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.704603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ef77f-80fa-460c-83c7-f4d4cf74387d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.704603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625ef77f-80fa-460c-83c7-f4d4cf74387d None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/users/25a17c1158c94c229e16c77c070bd9d7/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.706505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ef77f-80fa-460c-83c7-f4d4cf74387d None admin] RBAC: Authorizing `identity:create_grant(project_id=2c95a63dd5554fe1b35750e8ec50d808, user_id=25a17c1158c94c229e16c77c070bd9d7, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:53.707967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625ef77f-80fa-460c-83c7-f4d4cf74387d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:53.728531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 67/130] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 13:59:53 2022] PUT /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/users/25a17c1158c94c229e16c77c070bd9d7/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 13:59:53.729817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ef77f-80fa-460c-83c7-f4d4cf74387d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:53.763751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216d293-481d-49cd-a097-23a80aa23ef3 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.817714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e19265-f624-4d21-83bd-d901a64e91db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.819295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e19265-f624-4d21-83bd-d901a64e91db None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.819936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e19265-f624-4d21-83bd-d901a64e91db None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.820218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e19265-f624-4d21-83bd-d901a64e91db None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.821480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e19265-f624-4d21-83bd-d901a64e91db None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:53.823107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e19265-f624-4d21-83bd-d901a64e91db None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:53.825403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 64/131] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 13:59:53 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:53.831832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e19265-f624-4d21-83bd-d901a64e91db None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:53.907671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625ef77f-80fa-460c-83c7-f4d4cf74387d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.909488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:53.974048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e19265-f624-4d21-83bd-d901a64e91db None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:53.988970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762c7f0-176d-4982-8d63-a36d1fa49e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:53.989645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c7f0-176d-4982-8d63-a36d1fa49e1c None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:53.989927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c7f0-176d-4982-8d63-a36d1fa49e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:53.990928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762c7f0-176d-4982-8d63-a36d1fa49e1c None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/users/a072f8abd5f2437cbc748eae7f1f4f83/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:53.993730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762c7f0-176d-4982-8d63-a36d1fa49e1c None admin] RBAC: Authorizing `identity:create_grant(project_id=2c95a63dd5554fe1b35750e8ec50d808, user_id=a072f8abd5f2437cbc748eae7f1f4f83, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:53.995237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762c7f0-176d-4982-8d63-a36d1fa49e1c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:54.017665 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 68/132] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 13:59:53 2022] PUT /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/users/a072f8abd5f2437cbc748eae7f1f4f83/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 13:59:54.018390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762c7f0-176d-4982-8d63-a36d1fa49e1c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:54.020217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15b25f-c8fc-48ba-bad4-b603f67a8fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:54.020892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b25f-c8fc-48ba-bad4-b603f67a8fda None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:54.022065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b25f-c8fc-48ba-bad4-b603f67a8fda None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:54.022375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea15b25f-c8fc-48ba-bad4-b603f67a8fda None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:54.027471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15b25f-c8fc-48ba-bad4-b603f67a8fda None admin] RBAC: Adding query filter params (name=member) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:54.027471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15b25f-c8fc-48ba-bad4-b603f67a8fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:54.027471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea15b25f-c8fc-48ba-bad4-b603f67a8fda None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:54.027840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:54.040556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 65/133] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 13:59:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:54.046048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15b25f-c8fc-48ba-bad4-b603f67a8fda None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:54.193479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762c7f0-176d-4982-8d63-a36d1fa49e1c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:54.199239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea15b25f-c8fc-48ba-bad4-b603f67a8fda None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:54.270285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fd671-780b-4c79-85d3-69aeea8f3de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:54.270976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd671-780b-4c79-85d3-69aeea8f3de2 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:54.271359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd671-780b-4c79-85d3-69aeea8f3de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:54.271824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fd671-780b-4c79-85d3-69aeea8f3de2 None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/users/a072f8abd5f2437cbc748eae7f1f4f83/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:54.275332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fd671-780b-4c79-85d3-69aeea8f3de2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c95a63dd5554fe1b35750e8ec50d808, user_id=a072f8abd5f2437cbc748eae7f1f4f83, role_id=9c8bfff8d1a843dbb8aa6482c9fa35bf)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:54.276884 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fd671-780b-4c79-85d3-69aeea8f3de2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:54.281506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e575b7-7cc2-4d96-8f8d-f9e97c963cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:54.282168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e575b7-7cc2-4d96-8f8d-f9e97c963cb3 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:54.282452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e575b7-7cc2-4d96-8f8d-f9e97c963cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:54.282732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e575b7-7cc2-4d96-8f8d-f9e97c963cb3 None admin] PATH_INFO: `/v3/users/a072f8abd5f2437cbc748eae7f1f4f83` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:54.283952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e575b7-7cc2-4d96-8f8d-f9e97c963cb3 None admin] RBAC: Authorizing `identity:get_user(user_id=a072f8abd5f2437cbc748eae7f1f4f83)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:54.285591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e575b7-7cc2-4d96-8f8d-f9e97c963cb3 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:54.288019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 66/134] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:54 2022] GET /identity/v3/users/a072f8abd5f2437cbc748eae7f1f4f83 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 13:59:54.294648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e575b7-7cc2-4d96-8f8d-f9e97c963cb3 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:54.309843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 69/135] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 13:59:54 2022] PUT /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/users/a072f8abd5f2437cbc748eae7f1f4f83/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 13:59:54.311270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ca2d1-00ec-4428-86d5-585f74352de1 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:54.311608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ca2d1-00ec-4428-86d5-585f74352de1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:54.311980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069ca2d1-00ec-4428-86d5-585f74352de1 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:54.313702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 70/136] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:54.320315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad79c5-2899-4beb-b44c-7012170ea52a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:54.320662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad79c5-2899-4beb-b44c-7012170ea52a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:54.320943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad79c5-2899-4beb-b44c-7012170ea52a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:54.341964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7fad79c5-2899-4beb-b44c-7012170ea52a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:54.343211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7fad79c5-2899-4beb-b44c-7012170ea52a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:54.367093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:54.401789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:54.469789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 71/137] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:54.484496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e575b7-7cc2-4d96-8f8d-f9e97c963cb3 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:54.541945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d27eca-e93d-4722-a27a-5631c48a253f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:54.542599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d27eca-e93d-4722-a27a-5631c48a253f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:54.542881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d27eca-e93d-4722-a27a-5631c48a253f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:54.543159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d27eca-e93d-4722-a27a-5631c48a253f None admin] PATH_INFO: `/v3/projects/12ccf7fa23154beeb220ad57780c582e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:54.544342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d27eca-e93d-4722-a27a-5631c48a253f None admin] RBAC: Authorizing `identity:get_project(project_id=12ccf7fa23154beeb220ad57780c582e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:54.545801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d27eca-e93d-4722-a27a-5631c48a253f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:54.547854 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 67/138] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 13:59:54 2022] GET /identity/v3/projects/12ccf7fa23154beeb220ad57780c582e => generated 312 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:54.556535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad79c5-2899-4beb-b44c-7012170ea52a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:54.681671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad79c5-2899-4beb-b44c-7012170ea52a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:54.732001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1304627-70c9-4aa8-bffd-3e52e6113ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:54.732595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1304627-70c9-4aa8-bffd-3e52e6113ba1 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:54.732870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1304627-70c9-4aa8-bffd-3e52e6113ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:54.733150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1304627-70c9-4aa8-bffd-3e52e6113ba1 None admin] PATH_INFO: `/v3/projects/12ccf7fa23154beeb220ad57780c582e/users/a072f8abd5f2437cbc748eae7f1f4f83/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:54.739840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1304627-70c9-4aa8-bffd-3e52e6113ba1 None admin] RBAC: Authorizing `identity:create_grant(project_id=12ccf7fa23154beeb220ad57780c582e, user_id=a072f8abd5f2437cbc748eae7f1f4f83, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:54.742177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1304627-70c9-4aa8-bffd-3e52e6113ba1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:54.761324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 72/139] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 13:59:54 2022] PUT /identity/v3/projects/12ccf7fa23154beeb220ad57780c582e/users/a072f8abd5f2437cbc748eae7f1f4f83/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 13:59:54.933568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d27eca-e93d-4722-a27a-5631c48a253f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:55.069496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d27eca-e93d-4722-a27a-5631c48a253f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:55.102808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:55.111869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c081f12-b82d-4cd8-9373-387d7a72e2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:55.112723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c081f12-b82d-4cd8-9373-387d7a72e2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:55.113019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c081f12-b82d-4cd8-9373-387d7a72e2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:55.113388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c081f12-b82d-4cd8-9373-387d7a72e2b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:55.114737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c081f12-b82d-4cd8-9373-387d7a72e2b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:55.116746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c081f12-b82d-4cd8-9373-387d7a72e2b2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:55.118970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 68/140] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 13:59:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:55.125798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1304627-70c9-4aa8-bffd-3e52e6113ba1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:55.148320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:55.246833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1304627-70c9-4aa8-bffd-3e52e6113ba1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:55.295830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0b375-11a9-4daf-bb25-f3861312deeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:55.296444 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0b375-11a9-4daf-bb25-f3861312deeb None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:55.296725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0b375-11a9-4daf-bb25-f3861312deeb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:55.297001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0b375-11a9-4daf-bb25-f3861312deeb None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:55.297663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0b375-11a9-4daf-bb25-f3861312deeb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:55.299135 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0b375-11a9-4daf-bb25-f3861312deeb None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:55.337216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 73/141] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 13:59:55 2022] POST /identity/v3/users => generated 283 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 13:59:56.948505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad1291-a7cf-41ff-8cd5-149df3195c77 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:56.949055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad1291-a7cf-41ff-8cd5-149df3195c77 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:56.949055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fad1291-a7cf-41ff-8cd5-149df3195c77 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:56.950401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 69/142] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:56.961269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7c1b2-f831-456f-87ca-e89bf8ece23e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:56.961793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7c1b2-f831-456f-87ca-e89bf8ece23e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:56.961793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7c1b2-f831-456f-87ca-e89bf8ece23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:56.988409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f4c7c1b2-f831-456f-87ca-e89bf8ece23e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:56.989642 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f4c7c1b2-f831-456f-87ca-e89bf8ece23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:57.061653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:57.067492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 74/143] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:57.190643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909815c-9d11-4adc-ba78-e80ce641bd64 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:57.191034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909815c-9d11-4adc-ba78-e80ce641bd64 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:57.191413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b909815c-9d11-4adc-ba78-e80ce641bd64 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:57.193510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 70/144] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:57.201729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d958-cfb2-4a55-a264-d550518113b6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:57.202081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d958-cfb2-4a55-a264-d550518113b6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:57.202480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674d958-cfb2-4a55-a264-d550518113b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:57.232676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b674d958-cfb2-4a55-a264-d550518113b6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:57.233903 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b674d958-cfb2-4a55-a264-d550518113b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:57.317989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 75/145] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:57.477734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909815c-9d11-4adc-ba78-e80ce641bd64 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:57.524242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd7cc-931a-40bc-b46c-505ab5660c15 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:57.524551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd7cc-931a-40bc-b46c-505ab5660c15 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:57.524799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bcd7cc-931a-40bc-b46c-505ab5660c15 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:57.526221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 76/146] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:57.532153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec792d61-0704-4ec5-853a-5f17309bd22e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:57.532440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec792d61-0704-4ec5-853a-5f17309bd22e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:57.532715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec792d61-0704-4ec5-853a-5f17309bd22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:57.564152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ec792d61-0704-4ec5-853a-5f17309bd22e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:57.565358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ec792d61-0704-4ec5-853a-5f17309bd22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:57.672177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b909815c-9d11-4adc-ba78-e80ce641bd64 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:57.687526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 77/147] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:57.688811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa9052-f1ff-4cfd-8c57-d04259e9b1cd None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:57.689091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa9052-f1ff-4cfd-8c57-d04259e9b1cd None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:57.689360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fa9052-f1ff-4cfd-8c57-d04259e9b1cd None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:57.690586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 78/148] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:57.696947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4fbb-b781-4785-a409-909e480dec8d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:57.697221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4fbb-b781-4785-a409-909e480dec8d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:57.697493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb4fbb-b781-4785-a409-909e480dec8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:57.719295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee6c63-6906-41ff-af33-91a56b676559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:57.720054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee6c63-6906-41ff-af33-91a56b676559 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:57.720340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee6c63-6906-41ff-af33-91a56b676559 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:57.720631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee6c63-6906-41ff-af33-91a56b676559 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:57.726376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c2cb4fbb-b781-4785-a409-909e480dec8d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:57.727618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c2cb4fbb-b781-4785-a409-909e480dec8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:57.738560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ee6c63-6906-41ff-af33-91a56b676559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-09ee6c63-6906-41ff-af33-91a56b676559 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:57.744413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:57.745289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 13:59:57.746497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 71/149] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:57.750990 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee6c63-6906-41ff-af33-91a56b676559 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:57.796972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:57.810519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 79/150] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 13:59:57.812148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb4fbb-b781-4785-a409-909e480dec8d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:57.888760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:57.930279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb4fbb-b781-4785-a409-909e480dec8d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:57.953424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee6c63-6906-41ff-af33-91a56b676559 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:57.969252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084209b-6b4b-471c-bc75-55ec33774e4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:57.970639 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084209b-6b4b-471c-bc75-55ec33774e4d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:57.971229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084209b-6b4b-471c-bc75-55ec33774e4d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:57.971722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084209b-6b4b-471c-bc75-55ec33774e4d None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:57.972583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084209b-6b4b-471c-bc75-55ec33774e4d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:57.973373 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084209b-6b4b-471c-bc75-55ec33774e4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:57.976300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d084209b-6b4b-471c-bc75-55ec33774e4d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:57.985368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 80/151] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 13:59:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:57.991569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084209b-6b4b-471c-bc75-55ec33774e4d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:58.056191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f7b00-169e-49b7-9c3d-83ea08104a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:58.056848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f7b00-169e-49b7-9c3d-83ea08104a12 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:58.057131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f7b00-169e-49b7-9c3d-83ea08104a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:58.057407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f7b00-169e-49b7-9c3d-83ea08104a12 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:58.084043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:58.116181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084209b-6b4b-471c-bc75-55ec33774e4d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:58.120135 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f7b00-169e-49b7-9c3d-83ea08104a12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-fc5f7b00-169e-49b7-9c3d-83ea08104a12 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:58.127795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:58.128872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 13:59:58.129912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 72/152] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:58.138410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f7b00-169e-49b7-9c3d-83ea08104a12 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:58.228170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3c1c2-984f-4dcd-88f4-a222d8519186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:58.228816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3c1c2-984f-4dcd-88f4-a222d8519186 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:58.229095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3c1c2-984f-4dcd-88f4-a222d8519186 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:58.229372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c3c1c2-984f-4dcd-88f4-a222d8519186 None admin] PATH_INFO: `/v3/users/c6ddcc76426246a08038d84953728f4c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:58.230562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3c1c2-984f-4dcd-88f4-a222d8519186 None admin] RBAC: Authorizing `identity:get_user(user_id=c6ddcc76426246a08038d84953728f4c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:58.232131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c3c1c2-984f-4dcd-88f4-a222d8519186 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:58.233739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 81/153] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:57 2022] GET /identity/v3/users/c6ddcc76426246a08038d84953728f4c => generated 289 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:58.234504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3c1c2-984f-4dcd-88f4-a222d8519186 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:58.421841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f7b00-169e-49b7-9c3d-83ea08104a12 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:58.478537 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:58.513430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c3c1c2-984f-4dcd-88f4-a222d8519186 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:58.537562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:58.544289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18b55e-38ba-4b62-94cf-458aa0c068ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:58.544928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18b55e-38ba-4b62-94cf-458aa0c068ce None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:58.545212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18b55e-38ba-4b62-94cf-458aa0c068ce None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:58.545489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff18b55e-38ba-4b62-94cf-458aa0c068ce None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:58.546173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18b55e-38ba-4b62-94cf-458aa0c068ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:58.546479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18b55e-38ba-4b62-94cf-458aa0c068ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:58.551862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff18b55e-38ba-4b62-94cf-458aa0c068ce None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:58.565470 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e6720-0a6c-4e96-b796-7f77b6b3048c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:58.566130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e6720-0a6c-4e96-b796-7f77b6b3048c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:58.566403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e6720-0a6c-4e96-b796-7f77b6b3048c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:58.566734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e6720-0a6c-4e96-b796-7f77b6b3048c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:58.577475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 73/154] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 13:59:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 439 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 13:59:58.578359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18b55e-38ba-4b62-94cf-458aa0c068ce None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:58.587975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e6720-0a6c-4e96-b796-7f77b6b3048c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-db9e6720-0a6c-4e96-b796-7f77b6b3048c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:58.600117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:58.601098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 82/155] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 13:59:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:58.602490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e6720-0a6c-4e96-b796-7f77b6b3048c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:58.777876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e6720-0a6c-4e96-b796-7f77b6b3048c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:58.818212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff18b55e-38ba-4b62-94cf-458aa0c068ce None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:58.863022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96982250-e202-45cf-9376-a9811883faa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:58.864488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96982250-e202-45cf-9376-a9811883faa8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:58.866210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96982250-e202-45cf-9376-a9811883faa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:58.867188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96982250-e202-45cf-9376-a9811883faa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:58.871242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7622-b184-4773-9892-0cb8ab1a6aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:58.872420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7622-b184-4773-9892-0cb8ab1a6aca None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:58.873642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7622-b184-4773-9892-0cb8ab1a6aca None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:58.874358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3d7622-b184-4773-9892-0cb8ab1a6aca None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:58.878078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d7622-b184-4773-9892-0cb8ab1a6aca None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:58.879945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3d7622-b184-4773-9892-0cb8ab1a6aca None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:58.882282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 74/156] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 13:59:58 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:58.884451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7622-b184-4773-9892-0cb8ab1a6aca None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:58.894200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96982250-e202-45cf-9376-a9811883faa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-96982250-e202-45cf-9376-a9811883faa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 13:59:58.899578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 13:59:58.900390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 13:59:58.901131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 13:59:58.904794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 83/157] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 13:59:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 13:59:58.905595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96982250-e202-45cf-9376-a9811883faa8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:58.981247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:59.001686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:59.050266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3d7622-b184-4773-9892-0cb8ab1a6aca None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:59.085187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e55ec2-8607-480a-95ee-bf38a3ee1028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:59.086183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e55ec2-8607-480a-95ee-bf38a3ee1028 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:59.086665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e55ec2-8607-480a-95ee-bf38a3ee1028 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:59.087127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e55ec2-8607-480a-95ee-bf38a3ee1028 None admin] PATH_INFO: `/v3/users/a072f8abd5f2437cbc748eae7f1f4f83` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:59.088487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e55ec2-8607-480a-95ee-bf38a3ee1028 None admin] RBAC: Authorizing `identity:get_user(user_id=a072f8abd5f2437cbc748eae7f1f4f83)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:59.090545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e55ec2-8607-480a-95ee-bf38a3ee1028 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:59.092367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 75/158] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:58 2022] GET /identity/v3/users/a072f8abd5f2437cbc748eae7f1f4f83 => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:59.094074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46db3cc-1084-4a81-91b6-205ae6eb1151 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:59.094559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46db3cc-1084-4a81-91b6-205ae6eb1151 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:59.095020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46db3cc-1084-4a81-91b6-205ae6eb1151 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:59.096437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 76/159] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 13:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 13:59:59.097882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46db3cc-1084-4a81-91b6-205ae6eb1151 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:59.151472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96982250-e202-45cf-9376-a9811883faa8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:59.212141 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46db3cc-1084-4a81-91b6-205ae6eb1151 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:59.213944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd189a-802f-48b8-96fb-8c193c3c5a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:59.214558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd189a-802f-48b8-96fb-8c193c3c5a66 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:59.214898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd189a-802f-48b8-96fb-8c193c3c5a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:59.215173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cd189a-802f-48b8-96fb-8c193c3c5a66 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:59.216271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd189a-802f-48b8-96fb-8c193c3c5a66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:59.216271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd189a-802f-48b8-96fb-8c193c3c5a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:59.217730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cd189a-802f-48b8-96fb-8c193c3c5a66 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:59.233777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:59.241993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 84/160] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 13:59:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:59.247329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd189a-802f-48b8-96fb-8c193c3c5a66 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:59.265576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a38063e-1ade-49b2-a8f2-d14f82960cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:59.266612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38063e-1ade-49b2-a8f2-d14f82960cdd None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:59.267733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38063e-1ade-49b2-a8f2-d14f82960cdd None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:59.269148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a38063e-1ade-49b2-a8f2-d14f82960cdd None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/users/c6ddcc76426246a08038d84953728f4c/roles/efc3726869ca4763809656e2e7e1646a` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:59.273794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38063e-1ade-49b2-a8f2-d14f82960cdd None admin] RBAC: Authorizing `identity:create_grant(project_id=2c95a63dd5554fe1b35750e8ec50d808, user_id=c6ddcc76426246a08038d84953728f4c, role_id=efc3726869ca4763809656e2e7e1646a)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:59.276212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a38063e-1ade-49b2-a8f2-d14f82960cdd None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:59.298969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 77/161] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 13:59:59 2022] PUT /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/users/c6ddcc76426246a08038d84953728f4c/roles/efc3726869ca4763809656e2e7e1646a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 13:59:59.301952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d56880-f771-47e1-9fd5-0a050b55c627 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:59.309531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d56880-f771-47e1-9fd5-0a050b55c627 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:59.310040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d56880-f771-47e1-9fd5-0a050b55c627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:59.355176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-50d56880-f771-47e1-9fd5-0a050b55c627 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 13:59:59.361397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-50d56880-f771-47e1-9fd5-0a050b55c627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 13:59:59.459826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:59.462613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cd189a-802f-48b8-96fb-8c193c3c5a66 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:59.518808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb06118-aa76-4d2d-8366-3b8a21eef0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:59.519507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb06118-aa76-4d2d-8366-3b8a21eef0da None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:59.519786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb06118-aa76-4d2d-8366-3b8a21eef0da None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:59.520053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb06118-aa76-4d2d-8366-3b8a21eef0da None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:59.520779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb06118-aa76-4d2d-8366-3b8a21eef0da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 13:59:59.521110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb06118-aa76-4d2d-8366-3b8a21eef0da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:59.523092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb06118-aa76-4d2d-8366-3b8a21eef0da None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:59.539212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 85/162] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 13:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:59.540283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb06118-aa76-4d2d-8366-3b8a21eef0da None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:59.582046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 78/163] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 13:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 13:59:59.582655 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d56880-f771-47e1-9fd5-0a050b55c627 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:59.687982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb06118-aa76-4d2d-8366-3b8a21eef0da None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:59.726938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c938c-c63f-4f96-bf92-b42a8e72c515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:59.727582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c938c-c63f-4f96-bf92-b42a8e72c515 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:59.727861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c938c-c63f-4f96-bf92-b42a8e72c515 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:59.728126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c938c-c63f-4f96-bf92-b42a8e72c515 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:59.782905 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:59.783240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d56880-f771-47e1-9fd5-0a050b55c627 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 13:59:59.796621 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 13:59:59.832463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e330b10-48e6-4847-837c-04af41fcb36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 13:59:59.833026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e330b10-48e6-4847-837c-04af41fcb36a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 13:59:59.833299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e330b10-48e6-4847-837c-04af41fcb36a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 13:59:59.833588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e330b10-48e6-4847-837c-04af41fcb36a None admin] PATH_INFO: `/v3/users/a072f8abd5f2437cbc748eae7f1f4f83` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 13:59:59.837994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e330b10-48e6-4847-837c-04af41fcb36a None admin] RBAC: Authorizing `identity:get_user(user_id=a072f8abd5f2437cbc748eae7f1f4f83)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:59.839456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e330b10-48e6-4847-837c-04af41fcb36a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 13:59:59.841066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 79/164] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:59 2022] GET /identity/v3/users/a072f8abd5f2437cbc748eae7f1f4f83 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 13:59:59.844219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e330b10-48e6-4847-837c-04af41fcb36a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 13:59:59.864423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c938c-c63f-4f96-bf92-b42a8e72c515 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 13:59:59.867000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c938c-c63f-4f96-bf92-b42a8e72c515 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:00.040798 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e330b10-48e6-4847-837c-04af41fcb36a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:00.067208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 86/165] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 13:59:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:00.070400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e8b44-677d-4a41-9649-1b5cfe3a72d2 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.070687 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e8b44-677d-4a41-9649-1b5cfe3a72d2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.070959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89e8b44-677d-4a41-9649-1b5cfe3a72d2 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.079670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 87/166] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:00 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:00.080507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e8b44-677d-4a41-9649-1b5cfe3a72d2 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:00.118906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790ac40-fb67-4e94-99e1-328bd3ebb9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:00.119506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790ac40-fb67-4e94-99e1-328bd3ebb9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.119789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790ac40-fb67-4e94-99e1-328bd3ebb9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.120170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0790ac40-fb67-4e94-99e1-328bd3ebb9d4 None admin] PATH_INFO: `/v3/users/25a17c1158c94c229e16c77c070bd9d7` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.123599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790ac40-fb67-4e94-99e1-328bd3ebb9d4 None admin] RBAC: Authorizing `identity:get_user(user_id=25a17c1158c94c229e16c77c070bd9d7)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:00.125316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0790ac40-fb67-4e94-99e1-328bd3ebb9d4 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:00.127140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 80/167] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 13:59:59 2022] GET /identity/v3/users/25a17c1158c94c229e16c77c070bd9d7 => generated 247 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:00.131487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790ac40-fb67-4e94-99e1-328bd3ebb9d4 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:00.176179 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:00.222341 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89e8b44-677d-4a41-9649-1b5cfe3a72d2 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:00.227804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:00.263288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b24f3-e228-47ef-bb65-53dae6ec221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:00.264241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b24f3-e228-47ef-bb65-53dae6ec221b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.264696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b24f3-e228-47ef-bb65-53dae6ec221b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.265141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b24f3-e228-47ef-bb65-53dae6ec221b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.352171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b24f3-e228-47ef-bb65-53dae6ec221b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:00.354120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b24f3-e228-47ef-bb65-53dae6ec221b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:00.360201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0790ac40-fb67-4e94-99e1-328bd3ebb9d4 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:00.425146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6662b-3655-422d-af03-e2c65bff4cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:00.425146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6662b-3655-422d-af03-e2c65bff4cfe None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.425146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6662b-3655-422d-af03-e2c65bff4cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.425146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6662b-3655-422d-af03-e2c65bff4cfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.426133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:00.439317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6662b-3655-422d-af03-e2c65bff4cfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-4cd6662b-3655-422d-af03-e2c65bff4cfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:00.445558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:00.446484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 81/168] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:00.448226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6662b-3655-422d-af03-e2c65bff4cfe None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:00.479516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 88/169] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:00.480561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea58a7e-db96-47fa-a05f-f05855f8c89c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.480844 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea58a7e-db96-47fa-a05f-f05855f8c89c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.481142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea58a7e-db96-47fa-a05f-f05855f8c89c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.500921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-eea58a7e-db96-47fa-a05f-f05855f8c89c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:00.502116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-eea58a7e-db96-47fa-a05f-f05855f8c89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:00.558646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6662b-3655-422d-af03-e2c65bff4cfe None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:00.560930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:00.577241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 89/170] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:00.579627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea58a7e-db96-47fa-a05f-f05855f8c89c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:00.611529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81dc775-ce84-49b8-a278-664abc639bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:00.612309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dc775-ce84-49b8-a278-664abc639bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.612590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dc775-ce84-49b8-a278-664abc639bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.612866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81dc775-ce84-49b8-a278-664abc639bd9 None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.614153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81dc775-ce84-49b8-a278-664abc639bd9 None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:00.616093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81dc775-ce84-49b8-a278-664abc639bd9 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:00.618024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 82/171] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:00 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:00.619168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6082af6-48ad-4c6a-987e-271d7383730f None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.619474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6082af6-48ad-4c6a-987e-271d7383730f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.619761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6082af6-48ad-4c6a-987e-271d7383730f None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.621203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 83/172] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:00.622162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3e76c-79f0-4346-a2db-023e9c289636 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.622446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3e76c-79f0-4346-a2db-023e9c289636 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.622715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc3e76c-79f0-4346-a2db-023e9c289636 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.629333 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 84/173] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:00.629333 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3e76c-79f0-4346-a2db-023e9c289636 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:00.697293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:00.707174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea58a7e-db96-47fa-a05f-f05855f8c89c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:00.747264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b48229-0999-4c8b-b519-31b13bbd286b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:00.748759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b48229-0999-4c8b-b519-31b13bbd286b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.749544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b48229-0999-4c8b-b519-31b13bbd286b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.750026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b48229-0999-4c8b-b519-31b13bbd286b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.761830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc3e76c-79f0-4346-a2db-023e9c289636 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:00.797536 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1211fe0-7c01-45dd-8a75-e42b5e119c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:00.798204 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1211fe0-7c01-45dd-8a75-e42b5e119c74 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.798487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1211fe0-7c01-45dd-8a75-e42b5e119c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.799049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1211fe0-7c01-45dd-8a75-e42b5e119c74 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.799507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1211fe0-7c01-45dd-8a75-e42b5e119c74 None admin] RBAC: Adding query filter params (name=member) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:00.799811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1211fe0-7c01-45dd-8a75-e42b5e119c74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:00.801193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1211fe0-7c01-45dd-8a75-e42b5e119c74 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:00.811517 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 85/174] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:00.812185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1211fe0-7c01-45dd-8a75-e42b5e119c74 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:00.832204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b48229-0999-4c8b-b519-31b13bbd286b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:00.834800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b48229-0999-4c8b-b519-31b13bbd286b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:00.853881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:00.925271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1211fe0-7c01-45dd-8a75-e42b5e119c74 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:00.932590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:00.943159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 90/175] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:00.944667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf93e4-bd33-4201-b11e-2d4233f5cca4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.945016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf93e4-bd33-4201-b11e-2d4233f5cca4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.945362 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cf93e4-bd33-4201-b11e-2d4233f5cca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.963542 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-43cf93e4-bd33-4201-b11e-2d4233f5cca4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:00.964880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-43cf93e4-bd33-4201-b11e-2d4233f5cca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:00.967053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c605-323c-44b6-9d26-bbc87eee6b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:00.971565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c605-323c-44b6-9d26-bbc87eee6b0e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:00.971565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c605-323c-44b6-9d26-bbc87eee6b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:00.971565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345c605-323c-44b6-9d26-bbc87eee6b0e None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:00.972338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345c605-323c-44b6-9d26-bbc87eee6b0e None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:00.973961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345c605-323c-44b6-9d26-bbc87eee6b0e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:00.976201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 86/176] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:00 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:00.976770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c605-323c-44b6-9d26-bbc87eee6b0e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:01.053461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 91/177] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:01.054797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc64ad-116a-468d-aa7a-4d793cdcbfc8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:01.055086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc64ad-116a-468d-aa7a-4d793cdcbfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:01.055362 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc64ad-116a-468d-aa7a-4d793cdcbfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:01.074233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-36fc64ad-116a-468d-aa7a-4d793cdcbfc8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:01.075425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-36fc64ad-116a-468d-aa7a-4d793cdcbfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:01.092377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345c605-323c-44b6-9d26-bbc87eee6b0e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:01.094727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:01.129797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef693ac-975e-4c28-998d-541b711d9145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:01.131379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef693ac-975e-4c28-998d-541b711d9145 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:01.132067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef693ac-975e-4c28-998d-541b711d9145 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:01.132531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef693ac-975e-4c28-998d-541b711d9145 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:01.133444 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef693ac-975e-4c28-998d-541b711d9145 None admin] RBAC: Adding query filter params (role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808, user.id=a072f8abd5f2437cbc748eae7f1f4f83) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:01.133968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef693ac-975e-4c28-998d-541b711d9145 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:01.135690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef693ac-975e-4c28-998d-541b711d9145 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:01.141856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 87/178] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:00 2022] GET /identity/v3/role_assignments?user.id=a072f8abd5f2437cbc748eae7f1f4f83&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:01.142990 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef693ac-975e-4c28-998d-541b711d9145 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:01.156472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 92/179] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:01.157764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc64ad-116a-468d-aa7a-4d793cdcbfc8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:01.294122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:01.357914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef693ac-975e-4c28-998d-541b711d9145 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:01.397985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc64ad-116a-468d-aa7a-4d793cdcbfc8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:01.419309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fdde2-ecb5-41bc-8c54-3cf0ab1462be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:01.419980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fdde2-ecb5-41bc-8c54-3cf0ab1462be None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:01.420255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fdde2-ecb5-41bc-8c54-3cf0ab1462be None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:01.420527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8fdde2-ecb5-41bc-8c54-3cf0ab1462be None admin] PATH_INFO: `/v3/users/a072f8abd5f2437cbc748eae7f1f4f83` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:01.424116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fdde2-ecb5-41bc-8c54-3cf0ab1462be None admin] RBAC: Authorizing `identity:get_user(user_id=a072f8abd5f2437cbc748eae7f1f4f83)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:01.425583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8fdde2-ecb5-41bc-8c54-3cf0ab1462be None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:01.428162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 88/180] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:01 2022] GET /identity/v3/users/a072f8abd5f2437cbc748eae7f1f4f83 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:01.429098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fdde2-ecb5-41bc-8c54-3cf0ab1462be None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:01.456620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d6a16-d9e0-48fc-8e00-eec55ea3b960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:01.457263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d6a16-d9e0-48fc-8e00-eec55ea3b960 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:01.457535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d6a16-d9e0-48fc-8e00-eec55ea3b960 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:01.457803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d6a16-d9e0-48fc-8e00-eec55ea3b960 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:01.478895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2d6a16-d9e0-48fc-8e00-eec55ea3b960 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-cb2d6a16-d9e0-48fc-8e00-eec55ea3b960 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:01.485670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:01.486538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:01.487345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:01.488517 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:01.489382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 93/181] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:00:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:01.491748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d6a16-d9e0-48fc-8e00-eec55ea3b960 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:01.575939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8fdde2-ecb5-41bc-8c54-3cf0ab1462be None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:01.613260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006ce45-dd14-4c33-b32f-13810cc55cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:01.613892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ce45-dd14-4c33-b32f-13810cc55cc8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:01.614170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ce45-dd14-4c33-b32f-13810cc55cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:01.614440 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4006ce45-dd14-4c33-b32f-13810cc55cc8 None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:01.616254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006ce45-dd14-4c33-b32f-13810cc55cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:01.617703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4006ce45-dd14-4c33-b32f-13810cc55cc8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:01.624052 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 89/182] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:01 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:01.624052 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006ce45-dd14-4c33-b32f-13810cc55cc8 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:01.633228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d6a16-d9e0-48fc-8e00-eec55ea3b960 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:01.647026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:01.673776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d643a-04bf-44e9-a286-9416acf2620d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:01.674411 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d643a-04bf-44e9-a286-9416acf2620d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:01.674706 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d643a-04bf-44e9-a286-9416acf2620d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:01.674959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245d643a-04bf-44e9-a286-9416acf2620d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:01.675720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d643a-04bf-44e9-a286-9416acf2620d None admin] RBAC: Adding query filter params (role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808, user.id=a072f8abd5f2437cbc748eae7f1f4f83) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:01.676029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d643a-04bf-44e9-a286-9416acf2620d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:01.678061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245d643a-04bf-44e9-a286-9416acf2620d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:01.684249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 94/183] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:01 2022] GET /identity/v3/role_assignments?user.id=a072f8abd5f2437cbc748eae7f1f4f83&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:01.691738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d643a-04bf-44e9-a286-9416acf2620d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:01.741564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4006ce45-dd14-4c33-b32f-13810cc55cc8 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:01.743581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:01.785977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57262cdd-82de-4e00-9226-54b95403eda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:01.787878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57262cdd-82de-4e00-9226-54b95403eda0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:01.788328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57262cdd-82de-4e00-9226-54b95403eda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:01.788766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57262cdd-82de-4e00-9226-54b95403eda0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:01.833433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245d643a-04bf-44e9-a286-9416acf2620d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:01.869332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57262cdd-82de-4e00-9226-54b95403eda0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:01.870820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57262cdd-82de-4e00-9226-54b95403eda0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:01.890161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21083b-2c4e-440c-9f44-64a7e0607446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:01.891158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21083b-2c4e-440c-9f44-64a7e0607446 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:01.891558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21083b-2c4e-440c-9f44-64a7e0607446 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:01.891902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21083b-2c4e-440c-9f44-64a7e0607446 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:01.903493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd21083b-2c4e-440c-9f44-64a7e0607446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-bd21083b-2c4e-440c-9f44-64a7e0607446 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:01.908960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:01.909820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:01.910578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 95/184] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:00:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:01.911288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21083b-2c4e-440c-9f44-64a7e0607446 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:01.962659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:02.017421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 90/185] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:02.017421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57262cdd-82de-4e00-9226-54b95403eda0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:02.018087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:02.041921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21083b-2c4e-440c-9f44-64a7e0607446 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:02.078270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae7baa-4943-40cd-8c89-bdba3415bdbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:02.078935 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae7baa-4943-40cd-8c89-bdba3415bdbc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.079141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae7baa-4943-40cd-8c89-bdba3415bdbc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.079410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae7baa-4943-40cd-8c89-bdba3415bdbc None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.080129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae7baa-4943-40cd-8c89-bdba3415bdbc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:02.080428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae7baa-4943-40cd-8c89-bdba3415bdbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:02.081812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae7baa-4943-40cd-8c89-bdba3415bdbc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:02.095697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 96/186] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:02.096496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae7baa-4943-40cd-8c89-bdba3415bdbc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:02.123299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57262cdd-82de-4e00-9226-54b95403eda0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:02.163832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135cc73-5090-482b-b640-89c7a99d9ee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:02.164471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135cc73-5090-482b-b640-89c7a99d9ee0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.164747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135cc73-5090-482b-b640-89c7a99d9ee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.165015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135cc73-5090-482b-b640-89c7a99d9ee0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.166107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135cc73-5090-482b-b640-89c7a99d9ee0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:02.167829 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135cc73-5090-482b-b640-89c7a99d9ee0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:02.171608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 91/187] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:02.171608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135cc73-5090-482b-b640-89c7a99d9ee0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:02.202751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:02.206776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae7baa-4943-40cd-8c89-bdba3415bdbc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:02.242393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d87124-1a19-4dd4-a4fe-3f66760d1ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:02.242998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d87124-1a19-4dd4-a4fe-3f66760d1ea5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.243273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d87124-1a19-4dd4-a4fe-3f66760d1ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.243574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d87124-1a19-4dd4-a4fe-3f66760d1ea5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.244307 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d87124-1a19-4dd4-a4fe-3f66760d1ea5 None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808, user.id=25a17c1158c94c229e16c77c070bd9d7) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:02.244609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d87124-1a19-4dd4-a4fe-3f66760d1ea5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:02.246077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d87124-1a19-4dd4-a4fe-3f66760d1ea5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:02.246745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:02.252078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 97/188] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:02 2022] GET /identity/v3/role_assignments?user.id=25a17c1158c94c229e16c77c070bd9d7&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:02.253065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d87124-1a19-4dd4-a4fe-3f66760d1ea5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:02.286108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135cc73-5090-482b-b640-89c7a99d9ee0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:02.327735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170bad3-4f35-425e-9846-8ebde2357323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:02.328360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170bad3-4f35-425e-9846-8ebde2357323 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.328639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170bad3-4f35-425e-9846-8ebde2357323 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.328905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170bad3-4f35-425e-9846-8ebde2357323 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.329589 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170bad3-4f35-425e-9846-8ebde2357323 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:02.329893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170bad3-4f35-425e-9846-8ebde2357323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:02.331306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170bad3-4f35-425e-9846-8ebde2357323 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:02.341103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 92/189] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:02.345020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170bad3-4f35-425e-9846-8ebde2357323 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:02.367521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d87124-1a19-4dd4-a4fe-3f66760d1ea5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:02.406482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b72f5-66f3-4066-99fb-853545d7b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:02.407530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b72f5-66f3-4066-99fb-853545d7b8ce None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.407869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b72f5-66f3-4066-99fb-853545d7b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.408204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75b72f5-66f3-4066-99fb-853545d7b8ce None admin] PATH_INFO: `/v3/projects/12ccf7fa23154beeb220ad57780c582e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.409453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b72f5-66f3-4066-99fb-853545d7b8ce None admin] RBAC: Authorizing `identity:get_project(project_id=12ccf7fa23154beeb220ad57780c582e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:02.411021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75b72f5-66f3-4066-99fb-853545d7b8ce None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:02.413065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 98/190] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:02 2022] GET /identity/v3/projects/12ccf7fa23154beeb220ad57780c582e => generated 312 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:02.413730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b72f5-66f3-4066-99fb-853545d7b8ce None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:02.457070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:02.477846 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170bad3-4f35-425e-9846-8ebde2357323 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:02.510978 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:02.517084 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9daf5-1f79-4b28-abca-99c4e0741138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:02.517774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9daf5-1f79-4b28-abca-99c4e0741138 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.518058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9daf5-1f79-4b28-abca-99c4e0741138 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.518329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9daf5-1f79-4b28-abca-99c4e0741138 None admin] PATH_INFO: `/v3/users/38f6e23b56464e44ac078817b44e4eec` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.533025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75b72f5-66f3-4066-99fb-853545d7b8ce None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:02.534442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9daf5-1f79-4b28-abca-99c4e0741138 None admin] RBAC: Authorizing `identity:get_user(user_id=38f6e23b56464e44ac078817b44e4eec)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:02.536035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c9daf5-1f79-4b28-abca-99c4e0741138 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:02.537843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 93/191] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:02 2022] GET /identity/v3/users/38f6e23b56464e44ac078817b44e4eec => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:02.539008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3d03b-8518-4bc8-ac9a-82cb7264cbb3 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.539289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3d03b-8518-4bc8-ac9a-82cb7264cbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.539633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a3d03b-8518-4bc8-ac9a-82cb7264cbb3 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.541010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 94/192] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:02.541545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3d03b-8518-4bc8-ac9a-82cb7264cbb3 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:02.566348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab098174-d344-4431-a69c-0b1bc9dd8aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:02.566950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab098174-d344-4431-a69c-0b1bc9dd8aa3 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.567223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab098174-d344-4431-a69c-0b1bc9dd8aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.567515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab098174-d344-4431-a69c-0b1bc9dd8aa3 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.568727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab098174-d344-4431-a69c-0b1bc9dd8aa3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:02.570151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab098174-d344-4431-a69c-0b1bc9dd8aa3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:02.600510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 99/193] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:00:02 2022] POST /identity/v3/users => generated 297 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:00:02.600510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab098174-d344-4431-a69c-0b1bc9dd8aa3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:02.672305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a3d03b-8518-4bc8-ac9a-82cb7264cbb3 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:02.691987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:02.709617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab098174-d344-4431-a69c-0b1bc9dd8aa3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:02.715163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6bffe2-8bc2-4dca-aea5-37181f6ebde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:02.716123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bffe2-8bc2-4dca-aea5-37181f6ebde6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.716584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bffe2-8bc2-4dca-aea5-37181f6ebde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.717034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6bffe2-8bc2-4dca-aea5-37181f6ebde6 None admin] PATH_INFO: `/v3/users/38f6e23b56464e44ac078817b44e4eec` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.718449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bffe2-8bc2-4dca-aea5-37181f6ebde6 None admin] RBAC: Authorizing `identity:get_user(user_id=38f6e23b56464e44ac078817b44e4eec)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:02.720508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6bffe2-8bc2-4dca-aea5-37181f6ebde6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:02.722439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 95/194] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:02 2022] GET /identity/v3/users/38f6e23b56464e44ac078817b44e4eec => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:02.723836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade290e-c935-4efa-8ede-fbd253758e8b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.724124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade290e-c935-4efa-8ede-fbd253758e8b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.724936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade290e-c935-4efa-8ede-fbd253758e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.745106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2ade290e-c935-4efa-8ede-fbd253758e8b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:02.746900 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2ade290e-c935-4efa-8ede-fbd253758e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:02.754524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8946cb-9940-4e01-9650-60d19c2b0248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:02.755151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8946cb-9940-4e01-9650-60d19c2b0248 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.756016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8946cb-9940-4e01-9650-60d19c2b0248 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.756249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8946cb-9940-4e01-9650-60d19c2b0248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.757030 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8946cb-9940-4e01-9650-60d19c2b0248 None admin] RBAC: Adding query filter params (role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, scope.project.id=12ccf7fa23154beeb220ad57780c582e, user.id=a072f8abd5f2437cbc748eae7f1f4f83) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:02.757378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8946cb-9940-4e01-9650-60d19c2b0248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:02.758947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8946cb-9940-4e01-9650-60d19c2b0248 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:02.765984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 100/195] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:02 2022] GET /identity/v3/role_assignments?user.id=a072f8abd5f2437cbc748eae7f1f4f83&scope.project.id=12ccf7fa23154beeb220ad57780c582e&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:02.766792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8946cb-9940-4e01-9650-60d19c2b0248 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:02.822373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:02.845153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 96/196] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:02.856456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade290e-c935-4efa-8ede-fbd253758e8b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:02.911578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8946cb-9940-4e01-9650-60d19c2b0248 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:02.961885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9bee4-cbd5-46ee-941f-22a9dfba23a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:02.962727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9bee4-cbd5-46ee-941f-22a9dfba23a8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:02.963153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9bee4-cbd5-46ee-941f-22a9dfba23a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:02.963632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9bee4-cbd5-46ee-941f-22a9dfba23a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:02.991172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:03.002628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade290e-c935-4efa-8ede-fbd253758e8b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:03.047267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba4abf-5298-4abb-9f5e-e2bbdfa896bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:03.047965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba4abf-5298-4abb-9f5e-e2bbdfa896bf None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:03.048203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba4abf-5298-4abb-9f5e-e2bbdfa896bf None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:03.048472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba4abf-5298-4abb-9f5e-e2bbdfa896bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:03.067082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9bee4-cbd5-46ee-941f-22a9dfba23a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:03.068979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9bee4-cbd5-46ee-941f-22a9dfba23a8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:03.122761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:03.137652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba4abf-5298-4abb-9f5e-e2bbdfa896bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:03.140212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba4abf-5298-4abb-9f5e-e2bbdfa896bf None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:03.187895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 101/197] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:03.193669 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9bee4-cbd5-46ee-941f-22a9dfba23a8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:03.248399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:03.263698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 97/198] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:03.264575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba4abf-5298-4abb-9f5e-e2bbdfa896bf None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:03.304064 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9bee4-cbd5-46ee-941f-22a9dfba23a8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:03.346852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57aa3c-80fc-4871-85ff-457891631239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:03.347527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57aa3c-80fc-4871-85ff-457891631239 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:03.347807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57aa3c-80fc-4871-85ff-457891631239 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:03.348076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db57aa3c-80fc-4871-85ff-457891631239 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:03.349910 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57aa3c-80fc-4871-85ff-457891631239 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:03.351583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db57aa3c-80fc-4871-85ff-457891631239 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:03.353538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 102/199] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:03 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:03.354316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57aa3c-80fc-4871-85ff-457891631239 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:03.370964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:03.387181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba4abf-5298-4abb-9f5e-e2bbdfa896bf None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:03.435962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf08dec-0bfd-4b76-b07a-e062b72dc471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:03.436608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf08dec-0bfd-4b76-b07a-e062b72dc471 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:03.436884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf08dec-0bfd-4b76-b07a-e062b72dc471 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:03.437155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf08dec-0bfd-4b76-b07a-e062b72dc471 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:03.450377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf08dec-0bfd-4b76-b07a-e062b72dc471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-daf08dec-0bfd-4b76-b07a-e062b72dc471 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:03.455427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:03.456315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:03.457093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 98/200] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:03.457808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf08dec-0bfd-4b76-b07a-e062b72dc471 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:03.480360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db57aa3c-80fc-4871-85ff-457891631239 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:03.523894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:03.532632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c8fb-0790-4dde-8b32-be671877fa2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:03.534509 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c8fb-0790-4dde-8b32-be671877fa2b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:03.535376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c8fb-0790-4dde-8b32-be671877fa2b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:03.536205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8c8fb-0790-4dde-8b32-be671877fa2b None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:03.538845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8c8fb-0790-4dde-8b32-be671877fa2b None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:03.540556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed8c8fb-0790-4dde-8b32-be671877fa2b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:03.543147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 103/201] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:03 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:03.544265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c8fb-0790-4dde-8b32-be671877fa2b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:03.572747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf08dec-0bfd-4b76-b07a-e062b72dc471 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:03.619498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bcba0-ca6f-4a83-939c-351f9a0a24c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:03.620634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bcba0-ca6f-4a83-939c-351f9a0a24c5 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:03.621233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bcba0-ca6f-4a83-939c-351f9a0a24c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:03.621581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bcba0-ca6f-4a83-939c-351f9a0a24c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:03.622382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bcba0-ca6f-4a83-939c-351f9a0a24c5 None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=87ea676d839141b4b3e827be73b4e648, user.id=38f6e23b56464e44ac078817b44e4eec) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:03.622762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bcba0-ca6f-4a83-939c-351f9a0a24c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:03.625136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bcba0-ca6f-4a83-939c-351f9a0a24c5 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:03.632967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 99/202] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:03 2022] GET /identity/v3/role_assignments?user.id=38f6e23b56464e44ac078817b44e4eec&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:03.633733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bcba0-ca6f-4a83-939c-351f9a0a24c5 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:03.674862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:03.701122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8c8fb-0790-4dde-8b32-be671877fa2b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:03.760870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decaefdc-657b-4098-8eb3-2d93a3ddfc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:03.761497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decaefdc-657b-4098-8eb3-2d93a3ddfc34 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:03.761772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decaefdc-657b-4098-8eb3-2d93a3ddfc34 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:03.762041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decaefdc-657b-4098-8eb3-2d93a3ddfc34 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:03.762725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decaefdc-657b-4098-8eb3-2d93a3ddfc34 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:03.763025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decaefdc-657b-4098-8eb3-2d93a3ddfc34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:03.764593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decaefdc-657b-4098-8eb3-2d93a3ddfc34 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:03.788305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 104/203] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:03.795141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decaefdc-657b-4098-8eb3-2d93a3ddfc34 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:03.858831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bcba0-ca6f-4a83-939c-351f9a0a24c5 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:03.865257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:03.902599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e7c32-ade2-4bf0-bf77-5ffaa812e3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:03.903212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7c32-ade2-4bf0-bf77-5ffaa812e3fc None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:03.903510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7c32-ade2-4bf0-bf77-5ffaa812e3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:03.903783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325e7c32-ade2-4bf0-bf77-5ffaa812e3fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:03.904508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e7c32-ade2-4bf0-bf77-5ffaa812e3fc None admin] RBAC: Adding query filter params (role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf, scope.project.id=87ea676d839141b4b3e827be73b4e648, user.id=38f6e23b56464e44ac078817b44e4eec) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:03.904813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e7c32-ade2-4bf0-bf77-5ffaa812e3fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:03.906249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325e7c32-ade2-4bf0-bf77-5ffaa812e3fc None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:03.912898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 100/204] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:03 2022] GET /identity/v3/role_assignments?user.id=38f6e23b56464e44ac078817b44e4eec&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:03.974155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decaefdc-657b-4098-8eb3-2d93a3ddfc34 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:04.031268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847db28e-8664-4cfe-a2bd-148a80d5aa77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:04.031957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847db28e-8664-4cfe-a2bd-148a80d5aa77 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:04.032233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847db28e-8664-4cfe-a2bd-148a80d5aa77 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:04.032504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847db28e-8664-4cfe-a2bd-148a80d5aa77 None admin] PATH_INFO: `/v3/users/c6ddcc76426246a08038d84953728f4c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:04.033680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847db28e-8664-4cfe-a2bd-148a80d5aa77 None admin] RBAC: Authorizing `identity:get_user(user_id=c6ddcc76426246a08038d84953728f4c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:04.035164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847db28e-8664-4cfe-a2bd-148a80d5aa77 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:04.037722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 105/205] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:03 2022] GET /identity/v3/users/c6ddcc76426246a08038d84953728f4c => generated 289 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:04.043683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e7c32-ade2-4bf0-bf77-5ffaa812e3fc None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:04.137705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:04.163175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325e7c32-ade2-4bf0-bf77-5ffaa812e3fc None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:04.198525 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1d8a5-1d60-4ba1-8119-34838e9efee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:04.199140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1d8a5-1d60-4ba1-8119-34838e9efee4 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:04.199414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1d8a5-1d60-4ba1-8119-34838e9efee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:04.199714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1d8a5-1d60-4ba1-8119-34838e9efee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:04.270076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1d8a5-1d60-4ba1-8119-34838e9efee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:04.271874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1d8a5-1d60-4ba1-8119-34838e9efee4 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:04.349348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:04.384706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 101/206] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:04.390802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847db28e-8664-4cfe-a2bd-148a80d5aa77 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:04.532297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847db28e-8664-4cfe-a2bd-148a80d5aa77 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:04.559183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:04.584705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d347810-20fb-4541-a93f-181053b34ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:04.585348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d347810-20fb-4541-a93f-181053b34ddb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:04.585633 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d347810-20fb-4541-a93f-181053b34ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:04.585901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d347810-20fb-4541-a93f-181053b34ddb None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:04.587055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d347810-20fb-4541-a93f-181053b34ddb None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:04.588554 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d347810-20fb-4541-a93f-181053b34ddb None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:04.590353 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 106/207] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:04 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:04.596423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1d8a5-1d60-4ba1-8119-34838e9efee4 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:04.706961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1d8a5-1d60-4ba1-8119-34838e9efee4 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:04.742764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a1b4f-9ae7-4c25-9790-b15e16775c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:04.743781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a1b4f-9ae7-4c25-9790-b15e16775c5e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:04.744260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a1b4f-9ae7-4c25-9790-b15e16775c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:04.744727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a1b4f-9ae7-4c25-9790-b15e16775c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:04.745669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a1b4f-9ae7-4c25-9790-b15e16775c5e None admin] RBAC: Adding query filter params (role.id=efc3726869ca4763809656e2e7e1646a, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808, user.id=c6ddcc76426246a08038d84953728f4c) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:04.746186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a1b4f-9ae7-4c25-9790-b15e16775c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:04.747849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a1b4f-9ae7-4c25-9790-b15e16775c5e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:04.754600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 102/208] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:04 2022] GET /identity/v3/role_assignments?user.id=c6ddcc76426246a08038d84953728f4c&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=efc3726869ca4763809656e2e7e1646a => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:05.237970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d222694-69ae-4f7b-a99d-92572534f303 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:05.238383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d222694-69ae-4f7b-a99d-92572534f303 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:05.238562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d222694-69ae-4f7b-a99d-92572534f303 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:05.240820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 107/209] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:05.246315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9420bf9-a377-4a2d-9b2d-1285ae4677e4 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:05.246764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9420bf9-a377-4a2d-9b2d-1285ae4677e4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:05.246934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9420bf9-a377-4a2d-9b2d-1285ae4677e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:05.270226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d9420bf9-a377-4a2d-9b2d-1285ae4677e4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:05.271994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d9420bf9-a377-4a2d-9b2d-1285ae4677e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:05.312548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:05.351195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 103/210] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:05.553590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9856-7762-43dd-bba2-203b35da86f1 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:05.554077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9856-7762-43dd-bba2-203b35da86f1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:05.554077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186c9856-7762-43dd-bba2-203b35da86f1 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:05.555369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 108/211] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:05.560825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fe407-c4f8-4e2b-b8c4-698d51a0d1e5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:05.561111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fe407-c4f8-4e2b-b8c4-698d51a0d1e5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:05.561385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fe407-c4f8-4e2b-b8c4-698d51a0d1e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:05.580016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d92fe407-c4f8-4e2b-b8c4-698d51a0d1e5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:05.581177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d92fe407-c4f8-4e2b-b8c4-698d51a0d1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:05.647569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 104/212] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:05.648651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9856-7762-43dd-bba2-203b35da86f1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:05.756715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186c9856-7762-43dd-bba2-203b35da86f1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:05.791741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b407ff9-ffd2-4963-8604-e19efecf1caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:05.792370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b407ff9-ffd2-4963-8604-e19efecf1caa None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:05.792650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b407ff9-ffd2-4963-8604-e19efecf1caa None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:05.792924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b407ff9-ffd2-4963-8604-e19efecf1caa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:05.804564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b407ff9-ffd2-4963-8604-e19efecf1caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-1b407ff9-ffd2-4963-8604-e19efecf1caa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:05.809624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:05.810496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:05.811257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 109/213] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:05.820539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fe407-c4f8-4e2b-b8c4-698d51a0d1e5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:05.921289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fe407-c4f8-4e2b-b8c4-698d51a0d1e5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:05.923745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:05.925725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b407ff9-ffd2-4963-8604-e19efecf1caa None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:05.965536 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28876f90-1f73-4715-8d27-a3398eda2cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:05.966178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28876f90-1f73-4715-8d27-a3398eda2cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:05.966460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28876f90-1f73-4715-8d27-a3398eda2cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:05.966737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28876f90-1f73-4715-8d27-a3398eda2cc5 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:05.967407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28876f90-1f73-4715-8d27-a3398eda2cc5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:05.967739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28876f90-1f73-4715-8d27-a3398eda2cc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:05.969379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28876f90-1f73-4715-8d27-a3398eda2cc5 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:05.979856 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:05.982682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 105/214] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:05.988373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28876f90-1f73-4715-8d27-a3398eda2cc5 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:06.041172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b407ff9-ffd2-4963-8604-e19efecf1caa None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:06.078483 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e05b6-26b4-488a-a89d-02da33f375e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:06.079085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e05b6-26b4-488a-a89d-02da33f375e2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.079367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e05b6-26b4-488a-a89d-02da33f375e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.079680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e05b6-26b4-488a-a89d-02da33f375e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:06.080808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e05b6-26b4-488a-a89d-02da33f375e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:06.082278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e05b6-26b4-488a-a89d-02da33f375e2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:06.084861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 110/215] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:06.090553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e05b6-26b4-488a-a89d-02da33f375e2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:06.112159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28876f90-1f73-4715-8d27-a3398eda2cc5 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:06.157094 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b900c-0a14-479a-b003-fbcc25fd93f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:06.158024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b900c-0a14-479a-b003-fbcc25fd93f7 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.158329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b900c-0a14-479a-b003-fbcc25fd93f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.158609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b900c-0a14-479a-b003-fbcc25fd93f7 None admin] PATH_INFO: `/v3/users/8b3701c605214ba5854ab5e003f9bd17` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:06.175677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b900c-0a14-479a-b003-fbcc25fd93f7 None admin] RBAC: Authorizing `identity:get_user(user_id=8b3701c605214ba5854ab5e003f9bd17)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:06.178542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b900c-0a14-479a-b003-fbcc25fd93f7 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:06.181238 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 106/216] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:05 2022] GET /identity/v3/users/8b3701c605214ba5854ab5e003f9bd17 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:06.186681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b900c-0a14-479a-b003-fbcc25fd93f7 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:06.210060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e05b6-26b4-488a-a89d-02da33f375e2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:06.214145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:06.228313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:06.255122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe8443-a3ca-4213-b70e-095c6e2f4381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:06.256451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe8443-a3ca-4213-b70e-095c6e2f4381 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.256854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe8443-a3ca-4213-b70e-095c6e2f4381 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.257197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe8443-a3ca-4213-b70e-095c6e2f4381 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:06.257961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe8443-a3ca-4213-b70e-095c6e2f4381 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:06.259994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbe8443-a3ca-4213-b70e-095c6e2f4381 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:06.295513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 111/217] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:00:06 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:00:06.317161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b900c-0a14-479a-b003-fbcc25fd93f7 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:06.340639 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab22e94-d0aa-48ed-b4f3-f6cde0f3a00b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.340966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab22e94-d0aa-48ed-b4f3-f6cde0f3a00b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.341130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab22e94-d0aa-48ed-b4f3-f6cde0f3a00b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:06.343519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 112/218] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:00:06.349683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d645d-3384-4f5d-ac47-b80955fe6fcc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.350015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d645d-3384-4f5d-ac47-b80955fe6fcc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.350297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04d645d-3384-4f5d-ac47-b80955fe6fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:06.362229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd198e-3163-4e81-be82-6b1dd42526ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:06.362970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd198e-3163-4e81-be82-6b1dd42526ee None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.363317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd198e-3163-4e81-be82-6b1dd42526ee None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.363625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd198e-3163-4e81-be82-6b1dd42526ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:06.374096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f04d645d-3384-4f5d-ac47-b80955fe6fcc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:06.375323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f04d645d-3384-4f5d-ac47-b80955fe6fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:06.454409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd198e-3163-4e81-be82-6b1dd42526ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:06.456028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd198e-3163-4e81-be82-6b1dd42526ee None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:06.482711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:06.489804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 113/219] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:06.569479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 107/220] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:06.575584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d645d-3384-4f5d-ac47-b80955fe6fcc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:06.585314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6cb40-b3fe-4364-80ac-e8c59fbaeab9 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.585589 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6cb40-b3fe-4364-80ac-e8c59fbaeab9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.585860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6cb40-b3fe-4364-80ac-e8c59fbaeab9 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:06.587355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 108/221] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:00:06.592364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c8282-6c72-4a32-bb91-44be01a7576a None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.592579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c8282-6c72-4a32-bb91-44be01a7576a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.592853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c8282-6c72-4a32-bb91-44be01a7576a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:06.611722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c04c8282-6c72-4a32-bb91-44be01a7576a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:06.612894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c04c8282-6c72-4a32-bb91-44be01a7576a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:06.702535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 109/222] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:06.706947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:06.711734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04d645d-3384-4f5d-ac47-b80955fe6fcc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:06.752532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fa316-f150-4e89-ac5d-95d10e3fc5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:06.753339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fa316-f150-4e89-ac5d-95d10e3fc5ca None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.753721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fa316-f150-4e89-ac5d-95d10e3fc5ca None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.754079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6fa316-f150-4e89-ac5d-95d10e3fc5ca None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:06.755356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fa316-f150-4e89-ac5d-95d10e3fc5ca None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:06.756931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6fa316-f150-4e89-ac5d-95d10e3fc5ca None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:06.758859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 114/223] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:06 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:06.765230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c8282-6c72-4a32-bb91-44be01a7576a None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:06.779273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fa316-f150-4e89-ac5d-95d10e3fc5ca None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:06.828963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:06.888561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c8282-6c72-4a32-bb91-44be01a7576a None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:06.903584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6fa316-f150-4e89-ac5d-95d10e3fc5ca None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:06.929933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb5eb8-0551-4476-8f2f-93850f49eda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:06.930805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5eb8-0551-4476-8f2f-93850f49eda0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.931255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5eb8-0551-4476-8f2f-93850f49eda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.931913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb5eb8-0551-4476-8f2f-93850f49eda0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:06.933399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb5eb8-0551-4476-8f2f-93850f49eda0 None admin] RBAC: Adding query filter params (role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, scope.project.id=87ea676d839141b4b3e827be73b4e648, user.id=8b3701c605214ba5854ab5e003f9bd17) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:06.934427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb5eb8-0551-4476-8f2f-93850f49eda0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:06.936409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb5eb8-0551-4476-8f2f-93850f49eda0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:06.950208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 110/224] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:06 2022] GET /identity/v3/role_assignments?user.id=8b3701c605214ba5854ab5e003f9bd17&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:06.989331 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbfe8f-eb7f-4312-aa7e-5ca13ccfd3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:06.990097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbfe8f-eb7f-4312-aa7e-5ca13ccfd3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:06.990377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbfe8f-eb7f-4312-aa7e-5ca13ccfd3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:06.990710 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbfe8f-eb7f-4312-aa7e-5ca13ccfd3d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:07.013947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbfe8f-eb7f-4312-aa7e-5ca13ccfd3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-47cbfe8f-eb7f-4312-aa7e-5ca13ccfd3d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:07.019044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:07.020004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:07.021812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 115/225] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:00:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:00:07.027002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb5eb8-0551-4476-8f2f-93850f49eda0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:07.055184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbfe8f-eb7f-4312-aa7e-5ca13ccfd3d6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:07.090312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:07.194738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbfe8f-eb7f-4312-aa7e-5ca13ccfd3d6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:07.211365 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb5eb8-0551-4476-8f2f-93850f49eda0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:07.229063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:07.233123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544488aa-11bf-464c-b770-f0dba0b70f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:07.234400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544488aa-11bf-464c-b770-f0dba0b70f0f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:07.234400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544488aa-11bf-464c-b770-f0dba0b70f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:07.234920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544488aa-11bf-464c-b770-f0dba0b70f0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:07.248174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544488aa-11bf-464c-b770-f0dba0b70f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:07.252061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849606bb-af80-45cf-82a2-69faeb52c1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:07.252745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849606bb-af80-45cf-82a2-69faeb52c1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:07.253028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849606bb-af80-45cf-82a2-69faeb52c1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:07.253300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849606bb-af80-45cf-82a2-69faeb52c1f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:07.253969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849606bb-af80-45cf-82a2-69faeb52c1f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:07.254272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849606bb-af80-45cf-82a2-69faeb52c1f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-544488aa-11bf-464c-b770-f0dba0b70f0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:07.255889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:07.256728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:07.257505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:07.258083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849606bb-af80-45cf-82a2-69faeb52c1f1 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:07.260202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 116/226] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:00:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:00:07.266806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544488aa-11bf-464c-b770-f0dba0b70f0f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:07.273173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 111/227] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:07.283500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849606bb-af80-45cf-82a2-69faeb52c1f1 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:07.439865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849606bb-af80-45cf-82a2-69faeb52c1f1 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:07.539948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1346ef-ea60-467e-94a8-0716c3bfbd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:07.540280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1346ef-ea60-467e-94a8-0716c3bfbd0e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:07.541355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1346ef-ea60-467e-94a8-0716c3bfbd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:07.541355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1346ef-ea60-467e-94a8-0716c3bfbd0e None admin] PATH_INFO: `/v3/users/38f6e23b56464e44ac078817b44e4eec` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:07.541973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1346ef-ea60-467e-94a8-0716c3bfbd0e None admin] RBAC: Authorizing `identity:get_user(user_id=38f6e23b56464e44ac078817b44e4eec)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:07.543430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1346ef-ea60-467e-94a8-0716c3bfbd0e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:07.545136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 112/228] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:07 2022] GET /identity/v3/users/38f6e23b56464e44ac078817b44e4eec => generated 283 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:07.554857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1346ef-ea60-467e-94a8-0716c3bfbd0e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:07.581498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:07.620628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:07.639291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544488aa-11bf-464c-b770-f0dba0b70f0f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:07.691496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1346ef-ea60-467e-94a8-0716c3bfbd0e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:07.747393 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02138f-a9d2-415c-aa41-9ff0d360fea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:07.748082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02138f-a9d2-415c-aa41-9ff0d360fea1 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:07.748360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02138f-a9d2-415c-aa41-9ff0d360fea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:07.748640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02138f-a9d2-415c-aa41-9ff0d360fea1 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:07.752864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02138f-a9d2-415c-aa41-9ff0d360fea1 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:07.754749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02138f-a9d2-415c-aa41-9ff0d360fea1 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:07.760101 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 113/229] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:07 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:07.768447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02138f-a9d2-415c-aa41-9ff0d360fea1 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:07.792286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1fe4a-7226-4878-909a-fc64489221c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:07.793352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1fe4a-7226-4878-909a-fc64489221c8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:07.793824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1fe4a-7226-4878-909a-fc64489221c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:07.794282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c1fe4a-7226-4878-909a-fc64489221c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:07.795151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1fe4a-7226-4878-909a-fc64489221c8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:07.795695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1fe4a-7226-4878-909a-fc64489221c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:07.797294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c1fe4a-7226-4878-909a-fc64489221c8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:07.822044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 117/230] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 556 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:07.832459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1fe4a-7226-4878-909a-fc64489221c8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:07.980386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02138f-a9d2-415c-aa41-9ff0d360fea1 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:07.995353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:08.042702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c1fe4a-7226-4878-909a-fc64489221c8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:08.060588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e1e90-e117-4fa3-b0fa-b5df178c9573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:08.061368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e1e90-e117-4fa3-b0fa-b5df178c9573 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:08.061657 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e1e90-e117-4fa3-b0fa-b5df178c9573 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:08.061930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9e1e90-e117-4fa3-b0fa-b5df178c9573 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648/users/38f6e23b56464e44ac078817b44e4eec/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:08.067640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e1e90-e117-4fa3-b0fa-b5df178c9573 None admin] RBAC: Authorizing `identity:create_grant(project_id=87ea676d839141b4b3e827be73b4e648, user_id=38f6e23b56464e44ac078817b44e4eec, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:08.069678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9e1e90-e117-4fa3-b0fa-b5df178c9573 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:08.095310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 114/231] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:00:07 2022] PUT /identity/v3/projects/87ea676d839141b4b3e827be73b4e648/users/38f6e23b56464e44ac078817b44e4eec/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:08.096572 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07a6c0-f207-46f1-b57d-c0b8fd096cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:08.097276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07a6c0-f207-46f1-b57d-c0b8fd096cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:08.097553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07a6c0-f207-46f1-b57d-c0b8fd096cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:08.097828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07a6c0-f207-46f1-b57d-c0b8fd096cd8 None admin] PATH_INFO: `/v3/users/38f6e23b56464e44ac078817b44e4eec` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:08.099028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07a6c0-f207-46f1-b57d-c0b8fd096cd8 None admin] RBAC: Authorizing `identity:get_user(user_id=38f6e23b56464e44ac078817b44e4eec)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:08.100774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07a6c0-f207-46f1-b57d-c0b8fd096cd8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:08.102813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 118/232] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:07 2022] GET /identity/v3/users/38f6e23b56464e44ac078817b44e4eec => generated 283 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:08.112800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e1e90-e117-4fa3-b0fa-b5df178c9573 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:08.260488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9e1e90-e117-4fa3-b0fa-b5df178c9573 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:08.299235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8750b2-0197-4a1d-8672-5ce525284544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:08.299867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8750b2-0197-4a1d-8672-5ce525284544 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:08.300148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8750b2-0197-4a1d-8672-5ce525284544 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:08.300423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8750b2-0197-4a1d-8672-5ce525284544 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:08.301559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8750b2-0197-4a1d-8672-5ce525284544 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:08.303617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8750b2-0197-4a1d-8672-5ce525284544 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:08.305409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 115/233] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:08 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:08.312204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07a6c0-f207-46f1-b57d-c0b8fd096cd8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:08.331612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:08.423275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07a6c0-f207-46f1-b57d-c0b8fd096cd8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:08.460504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb9e4a-0e90-4e66-bca9-72d2ab66aa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:08.461408 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9e4a-0e90-4e66-bca9-72d2ab66aa16 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:08.461839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9e4a-0e90-4e66-bca9-72d2ab66aa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:08.462293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb9e4a-0e90-4e66-bca9-72d2ab66aa16 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648/users/38f6e23b56464e44ac078817b44e4eec/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:08.464685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb9e4a-0e90-4e66-bca9-72d2ab66aa16 None admin] RBAC: Authorizing `identity:create_grant(project_id=87ea676d839141b4b3e827be73b4e648, user_id=38f6e23b56464e44ac078817b44e4eec, role_id=9c8bfff8d1a843dbb8aa6482c9fa35bf)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:08.466340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb9e4a-0e90-4e66-bca9-72d2ab66aa16 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:08.485221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 119/234] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:00:08 2022] PUT /identity/v3/projects/87ea676d839141b4b3e827be73b4e648/users/38f6e23b56464e44ac078817b44e4eec/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:09.373425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d474b8-7935-40ea-a773-4a4af00114a9 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:09.373425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d474b8-7935-40ea-a773-4a4af00114a9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:09.373425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d474b8-7935-40ea-a773-4a4af00114a9 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:09.374548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 116/235] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:09.378568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a5e6a-8404-45dd-b4e7-659de551d00b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:09.379351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a5e6a-8404-45dd-b4e7-659de551d00b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:09.379855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a5e6a-8404-45dd-b4e7-659de551d00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:09.401748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-259a5e6a-8404-45dd-b4e7-659de551d00b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:09.403259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-259a5e6a-8404-45dd-b4e7-659de551d00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:09.442611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f43446-e5ec-4d66-b3b5-5f240b69edec None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:09.443880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f43446-e5ec-4d66-b3b5-5f240b69edec None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:09.444370 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f43446-e5ec-4d66-b3b5-5f240b69edec None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:09.445911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 117/236] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:00:09.450953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d895a-2d60-4746-9fcd-57d7131e4e72 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:09.451469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d895a-2d60-4746-9fcd-57d7131e4e72 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:09.451964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d895a-2d60-4746-9fcd-57d7131e4e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:09.484119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-9e2d895a-2d60-4746-9fcd-57d7131e4e72 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:09.489067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9e2d895a-2d60-4746-9fcd-57d7131e4e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:09.539741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 120/237] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:09.559852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:09.591727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 118/238] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:09.599832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c382-377d-4c3f-bc0a-07ff7d91467f None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:09.600115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c382-377d-4c3f-bc0a-07ff7d91467f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:09.600387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26c382-377d-4c3f-bc0a-07ff7d91467f None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:09.601743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 121/239] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:09.607126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f9b52-47b1-4c7a-bf6b-94bb11ec94be None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:09.607409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f9b52-47b1-4c7a-bf6b-94bb11ec94be None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:09.607698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130f9b52-47b1-4c7a-bf6b-94bb11ec94be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:09.634361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-130f9b52-47b1-4c7a-bf6b-94bb11ec94be None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:09.635882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-130f9b52-47b1-4c7a-bf6b-94bb11ec94be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:09.735129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 119/240] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:09.894147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26c382-377d-4c3f-bc0a-07ff7d91467f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:09.903817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:09.937245 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f9b52-47b1-4c7a-bf6b-94bb11ec94be None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:10.085513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26c382-377d-4c3f-bc0a-07ff7d91467f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:10.104141 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:10.108554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130f9b52-47b1-4c7a-bf6b-94bb11ec94be None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:10.133813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f96371-0466-4aaa-886d-2c1e3bbc3d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:10.134435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f96371-0466-4aaa-886d-2c1e3bbc3d8a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:10.134722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f96371-0466-4aaa-886d-2c1e3bbc3d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:10.134982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f96371-0466-4aaa-886d-2c1e3bbc3d8a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:10.149086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f96371-0466-4aaa-886d-2c1e3bbc3d8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-15f96371-0466-4aaa-886d-2c1e3bbc3d8a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:10.154110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:10.154945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:10.155757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:10.156334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a78799-ea5f-4b46-8f08-c8fe8bf40367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:10.156334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a78799-ea5f-4b46-8f08-c8fe8bf40367 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:10.156334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a78799-ea5f-4b46-8f08-c8fe8bf40367 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:10.156334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a78799-ea5f-4b46-8f08-c8fe8bf40367 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:10.157063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a78799-ea5f-4b46-8f08-c8fe8bf40367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:10.158293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a78799-ea5f-4b46-8f08-c8fe8bf40367 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:10.158850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 122/241] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:10.163911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 120/242] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:10.164630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a78799-ea5f-4b46-8f08-c8fe8bf40367 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:10.165944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f96371-0466-4aaa-886d-2c1e3bbc3d8a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:10.237931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:10.276069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f96371-0466-4aaa-886d-2c1e3bbc3d8a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:10.288955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a78799-ea5f-4b46-8f08-c8fe8bf40367 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:10.313726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d27c9-ac20-4fe9-a018-e562524afdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:10.314741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d27c9-ac20-4fe9-a018-e562524afdb2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:10.315334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d27c9-ac20-4fe9-a018-e562524afdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:10.315843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d27c9-ac20-4fe9-a018-e562524afdb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:10.316699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d27c9-ac20-4fe9-a018-e562524afdb2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:10.317207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d27c9-ac20-4fe9-a018-e562524afdb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:10.319236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d27c9-ac20-4fe9-a018-e562524afdb2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:10.332611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 123/243] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:10.332787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d1629-38b9-4ca5-9dfe-3f2f0586bb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:10.332787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1629-38b9-4ca5-9dfe-3f2f0586bb48 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:10.332787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1629-38b9-4ca5-9dfe-3f2f0586bb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:10.332787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d1629-38b9-4ca5-9dfe-3f2f0586bb48 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:10.335159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d27c9-ac20-4fe9-a018-e562524afdb2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:10.347115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5d1629-38b9-4ca5-9dfe-3f2f0586bb48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-ef5d1629-38b9-4ca5-9dfe-3f2f0586bb48 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:10.353230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:10.354111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:10.354890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:10.356115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 121/244] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:10.356115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d1629-38b9-4ca5-9dfe-3f2f0586bb48 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:10.393606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:10.485705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d27c9-ac20-4fe9-a018-e562524afdb2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:10.487915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d1629-38b9-4ca5-9dfe-3f2f0586bb48 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:10.522858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:10.530574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056f8cd-a031-4011-98d6-022bf77ed8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:10.531314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056f8cd-a031-4011-98d6-022bf77ed8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:10.531614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056f8cd-a031-4011-98d6-022bf77ed8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:10.531942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a056f8cd-a031-4011-98d6-022bf77ed8d0 None admin] PATH_INFO: `/v3/users/99eea39b5aeb47d68a5d3cdf6d283861` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:10.534390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a1a80-3545-4835-bdb9-7843d9e0a3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:10.536028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a1a80-3545-4835-bdb9-7843d9e0a3a0 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:10.536308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a1a80-3545-4835-bdb9-7843d9e0a3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:10.536640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416a1a80-3545-4835-bdb9-7843d9e0a3a0 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:10.539606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a1a80-3545-4835-bdb9-7843d9e0a3a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:10.541418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416a1a80-3545-4835-bdb9-7843d9e0a3a0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:10.551401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056f8cd-a031-4011-98d6-022bf77ed8d0 None admin] RBAC: Authorizing `identity:get_user(user_id=99eea39b5aeb47d68a5d3cdf6d283861)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:10.554401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a056f8cd-a031-4011-98d6-022bf77ed8d0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:10.557169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 122/245] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:10 2022] GET /identity/v3/users/99eea39b5aeb47d68a5d3cdf6d283861 => generated 297 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:10.560407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056f8cd-a031-4011-98d6-022bf77ed8d0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:10.583927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 124/246] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:00:10 2022] POST /identity/v3/users => generated 293 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:00:10.585392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a1a80-3545-4835-bdb9-7843d9e0a3a0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:10.683569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:10.707163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a056f8cd-a031-4011-98d6-022bf77ed8d0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:10.790774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18245469-f977-41e7-b064-0c71d83a8e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:10.791413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18245469-f977-41e7-b064-0c71d83a8e32 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:10.791722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18245469-f977-41e7-b064-0c71d83a8e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:10.791995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18245469-f977-41e7-b064-0c71d83a8e32 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:10.792684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18245469-f977-41e7-b064-0c71d83a8e32 None admin] RBAC: Adding query filter params (name=member) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:10.792992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18245469-f977-41e7-b064-0c71d83a8e32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:10.794411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18245469-f977-41e7-b064-0c71d83a8e32 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:10.811521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 123/247] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:10.816895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18245469-f977-41e7-b064-0c71d83a8e32 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:10.826228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416a1a80-3545-4835-bdb9-7843d9e0a3a0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:10.864865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833334f7-977c-4cca-8794-ca62e901a54a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:10.866095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833334f7-977c-4cca-8794-ca62e901a54a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:10.866374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833334f7-977c-4cca-8794-ca62e901a54a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:10.866645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833334f7-977c-4cca-8794-ca62e901a54a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:10.935351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:10.960340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18245469-f977-41e7-b064-0c71d83a8e32 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:10.962559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833334f7-977c-4cca-8794-ca62e901a54a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:10.964194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833334f7-977c-4cca-8794-ca62e901a54a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:11.017041 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111a5187-0148-4e62-adfd-1abbe4e098a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:11.017679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a5187-0148-4e62-adfd-1abbe4e098a0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:11.017950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a5187-0148-4e62-adfd-1abbe4e098a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:11.018214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111a5187-0148-4e62-adfd-1abbe4e098a0 None admin] PATH_INFO: `/v3/users/8b3701c605214ba5854ab5e003f9bd17` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:11.019352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a5187-0148-4e62-adfd-1abbe4e098a0 None admin] RBAC: Authorizing `identity:get_user(user_id=8b3701c605214ba5854ab5e003f9bd17)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:11.021774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111a5187-0148-4e62-adfd-1abbe4e098a0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:11.023773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 124/248] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:10 2022] GET /identity/v3/users/8b3701c605214ba5854ab5e003f9bd17 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:11.027476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a694d62-03d7-4430-a24a-95b0bb7b983b None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:11.027476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a694d62-03d7-4430-a24a-95b0bb7b983b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:11.027476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a694d62-03d7-4430-a24a-95b0bb7b983b None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:11.027476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 125/249] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:11.029781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a694d62-03d7-4430-a24a-95b0bb7b983b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:11.061544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:11.105170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 125/250] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:11.106447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf533fa-2cdd-41a0-8e1f-657e8552a686 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:11.106728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf533fa-2cdd-41a0-8e1f-657e8552a686 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:11.106996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf533fa-2cdd-41a0-8e1f-657e8552a686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:11.129005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ddf533fa-2cdd-41a0-8e1f-657e8552a686 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:11.131331 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ddf533fa-2cdd-41a0-8e1f-657e8552a686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:11.181456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a694d62-03d7-4430-a24a-95b0bb7b983b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:11.218014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 126/251] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:11.219605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198e332-9c05-4017-8b85-fe0534e1af79 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:11.219888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198e332-9c05-4017-8b85-fe0534e1af79 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:11.220160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198e332-9c05-4017-8b85-fe0534e1af79 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:11.221963 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 127/252] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:11.222671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198e332-9c05-4017-8b85-fe0534e1af79 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:11.228388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d52c3a1-0a30-4651-85f9-2e8e1de494a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:11.229284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52c3a1-0a30-4651-85f9-2e8e1de494a3 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:11.229967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52c3a1-0a30-4651-85f9-2e8e1de494a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:11.230463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d52c3a1-0a30-4651-85f9-2e8e1de494a3 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:11.240761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52c3a1-0a30-4651-85f9-2e8e1de494a3 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:11.243310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d52c3a1-0a30-4651-85f9-2e8e1de494a3 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:11.250091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 126/253] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:11 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:11.256233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64789df6-b0d6-451f-8598-42cd52376ce9 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:11.258675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64789df6-b0d6-451f-8598-42cd52376ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:11.259234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64789df6-b0d6-451f-8598-42cd52376ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:11.266554 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:11.293774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-64789df6-b0d6-451f-8598-42cd52376ce9 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:11.294978 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-64789df6-b0d6-451f-8598-42cd52376ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:11.406607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198e332-9c05-4017-8b85-fe0534e1af79 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:11.446342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 127/254] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:11.448198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64789df6-b0d6-451f-8598-42cd52376ce9 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:11.479070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:11.492933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3881bb5-6d60-4c61-b7ea-38713f2fae58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:11.493676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881bb5-6d60-4c61-b7ea-38713f2fae58 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:11.493952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881bb5-6d60-4c61-b7ea-38713f2fae58 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:11.494221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3881bb5-6d60-4c61-b7ea-38713f2fae58 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:11.495624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3881bb5-6d60-4c61-b7ea-38713f2fae58 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:11.497787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3881bb5-6d60-4c61-b7ea-38713f2fae58 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:11.504773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 128/255] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:11 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:11.508262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3881bb5-6d60-4c61-b7ea-38713f2fae58 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:11.642603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64789df6-b0d6-451f-8598-42cd52376ce9 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:11.675537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:11.706785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3881bb5-6d60-4c61-b7ea-38713f2fae58 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:11.717042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50460120-8397-488a-838a-567ab5e7ba80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:11.717717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50460120-8397-488a-838a-567ab5e7ba80 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:11.717988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50460120-8397-488a-838a-567ab5e7ba80 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:11.718254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50460120-8397-488a-838a-567ab5e7ba80 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648/users/8b3701c605214ba5854ab5e003f9bd17/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:11.720498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50460120-8397-488a-838a-567ab5e7ba80 None admin] RBAC: Authorizing `identity:create_grant(project_id=87ea676d839141b4b3e827be73b4e648, user_id=8b3701c605214ba5854ab5e003f9bd17, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:11.722022 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50460120-8397-488a-838a-567ab5e7ba80 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:11.746942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 128/256] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:00:11 2022] PUT /identity/v3/projects/87ea676d839141b4b3e827be73b4e648/users/8b3701c605214ba5854ab5e003f9bd17/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:11.747689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50460120-8397-488a-838a-567ab5e7ba80 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:11.776878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80215f14-0def-4c26-ad3d-86891f409c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:11.777591 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80215f14-0def-4c26-ad3d-86891f409c68 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:11.777866 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80215f14-0def-4c26-ad3d-86891f409c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:11.778129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80215f14-0def-4c26-ad3d-86891f409c68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:11.791694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80215f14-0def-4c26-ad3d-86891f409c68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-80215f14-0def-4c26-ad3d-86891f409c68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:11.797037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:11.797920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:11.799147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 129/257] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:00:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:11.800046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80215f14-0def-4c26-ad3d-86891f409c68 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:11.853333 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:11.899733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50460120-8397-488a-838a-567ab5e7ba80 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:11.985284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909aca7-be54-4a73-8e14-3bbdff0c24a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:11.985913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909aca7-be54-4a73-8e14-3bbdff0c24a1 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:11.986190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909aca7-be54-4a73-8e14-3bbdff0c24a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:11.986463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909aca7-be54-4a73-8e14-3bbdff0c24a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:11.987204 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909aca7-be54-4a73-8e14-3bbdff0c24a1 None admin] RBAC: Adding query filter params (role.id=efc3726869ca4763809656e2e7e1646a, scope.project.id=87ea676d839141b4b3e827be73b4e648, user.id=99eea39b5aeb47d68a5d3cdf6d283861) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:11.987536 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909aca7-be54-4a73-8e14-3bbdff0c24a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:11.990434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909aca7-be54-4a73-8e14-3bbdff0c24a1 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:11.994396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80215f14-0def-4c26-ad3d-86891f409c68 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:12.007539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 129/258] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:11 2022] GET /identity/v3/role_assignments?user.id=99eea39b5aeb47d68a5d3cdf6d283861&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=efc3726869ca4763809656e2e7e1646a => generated 259 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:12.008310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909aca7-be54-4a73-8e14-3bbdff0c24a1 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:12.069369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3a130-920a-4f72-aeed-e3a82c068657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:12.070415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3a130-920a-4f72-aeed-e3a82c068657 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:12.070894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3a130-920a-4f72-aeed-e3a82c068657 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:12.071362 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3a130-920a-4f72-aeed-e3a82c068657 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:12.092694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3a130-920a-4f72-aeed-e3a82c068657 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-4cc3a130-920a-4f72-aeed-e3a82c068657 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:12.102906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:12.103883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:12.104738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:12.111077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 130/259] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:00:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:12.116701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3a130-920a-4f72-aeed-e3a82c068657 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:12.123818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:12.267385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909aca7-be54-4a73-8e14-3bbdff0c24a1 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:12.351859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3a130-920a-4f72-aeed-e3a82c068657 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:12.354764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24800d9-78d0-44f7-80d5-4544d3caaad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:12.355388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24800d9-78d0-44f7-80d5-4544d3caaad0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:12.355686 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24800d9-78d0-44f7-80d5-4544d3caaad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:12.355954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24800d9-78d0-44f7-80d5-4544d3caaad0 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:12.356623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24800d9-78d0-44f7-80d5-4544d3caaad0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:12.356924 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24800d9-78d0-44f7-80d5-4544d3caaad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:12.358324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24800d9-78d0-44f7-80d5-4544d3caaad0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:12.372912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:12.382247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 130/260] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:12.388297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24800d9-78d0-44f7-80d5-4544d3caaad0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:12.441800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f642a-334a-4c23-b3c5-10b53a4ea5bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:12.442233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f642a-334a-4c23-b3c5-10b53a4ea5bd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:12.442507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f642a-334a-4c23-b3c5-10b53a4ea5bd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:12.443323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f642a-334a-4c23-b3c5-10b53a4ea5bd None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:12.445365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f642a-334a-4c23-b3c5-10b53a4ea5bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:12.445678 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f642a-334a-4c23-b3c5-10b53a4ea5bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:12.447079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f642a-334a-4c23-b3c5-10b53a4ea5bd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:12.471046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 131/261] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:12.481229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f642a-334a-4c23-b3c5-10b53a4ea5bd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:12.606048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24800d9-78d0-44f7-80d5-4544d3caaad0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:12.638169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:12.679226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2980210-647e-43b2-812e-ea566878bf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:12.680201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2980210-647e-43b2-812e-ea566878bf5b None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:12.680894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2980210-647e-43b2-812e-ea566878bf5b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:12.681267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2980210-647e-43b2-812e-ea566878bf5b None admin] PATH_INFO: `/v3/users/38f6e23b56464e44ac078817b44e4eec` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:12.689026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2980210-647e-43b2-812e-ea566878bf5b None admin] RBAC: Authorizing `identity:get_user(user_id=38f6e23b56464e44ac078817b44e4eec)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:12.691231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2980210-647e-43b2-812e-ea566878bf5b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:12.694158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 131/262] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:12 2022] GET /identity/v3/users/38f6e23b56464e44ac078817b44e4eec => generated 283 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:12.700377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2980210-647e-43b2-812e-ea566878bf5b None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:12.731303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f642a-334a-4c23-b3c5-10b53a4ea5bd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:12.815472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:12.815660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d3de8-6716-4790-ac99-db0743365622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:12.815660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d3de8-6716-4790-ac99-db0743365622 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:12.815660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d3de8-6716-4790-ac99-db0743365622 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:12.815660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d3de8-6716-4790-ac99-db0743365622 None admin] PATH_INFO: `/v3/users/38f6e23b56464e44ac078817b44e4eec` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:12.816672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d3de8-6716-4790-ac99-db0743365622 None admin] RBAC: Authorizing `identity:get_user(user_id=38f6e23b56464e44ac078817b44e4eec)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:12.818538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d3de8-6716-4790-ac99-db0743365622 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:12.820214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 132/263] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:12 2022] GET /identity/v3/users/38f6e23b56464e44ac078817b44e4eec => generated 283 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:12.825763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d3de8-6716-4790-ac99-db0743365622 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:12.872851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2980210-647e-43b2-812e-ea566878bf5b None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:12.944221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81f946-6507-4fc2-b48b-14f2c06eed1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:12.945244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81f946-6507-4fc2-b48b-14f2c06eed1e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:12.945730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81f946-6507-4fc2-b48b-14f2c06eed1e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:12.946210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd81f946-6507-4fc2-b48b-14f2c06eed1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:13.061684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81f946-6507-4fc2-b48b-14f2c06eed1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:13.063182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd81f946-6507-4fc2-b48b-14f2c06eed1e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:13.073588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d3de8-6716-4790-ac99-db0743365622 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:13.132572 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:13.151024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016961bf-6db5-4b8c-9d80-4b91f0015e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:13.151673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016961bf-6db5-4b8c-9d80-4b91f0015e09 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:13.151949 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016961bf-6db5-4b8c-9d80-4b91f0015e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:13.152213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016961bf-6db5-4b8c-9d80-4b91f0015e09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:13.219724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:13.310512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 132/264] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:13.317212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81f946-6507-4fc2-b48b-14f2c06eed1e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:13.320074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016961bf-6db5-4b8c-9d80-4b91f0015e09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:13.321722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016961bf-6db5-4b8c-9d80-4b91f0015e09 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:13.533653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 133/265] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 708 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 14:00:13.540262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016961bf-6db5-4b8c-9d80-4b91f0015e09 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:13.548089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd81f946-6507-4fc2-b48b-14f2c06eed1e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:13.586775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:13.626199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4371f-a704-4257-ab51-1995c5a727b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:13.628460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4371f-a704-4257-ab51-1995c5a727b9 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:13.630094 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4371f-a704-4257-ab51-1995c5a727b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:13.630646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4371f-a704-4257-ab51-1995c5a727b9 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:13.633193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4371f-a704-4257-ab51-1995c5a727b9 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:13.635098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4371f-a704-4257-ab51-1995c5a727b9 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:13.637754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 133/266] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:13 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:13.643950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8916-8582-4371-9f39-839e68465394 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:13.648205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8916-8582-4371-9f39-839e68465394 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:13.650847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e8916-8582-4371-9f39-839e68465394 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:13.652495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 134/267] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:13 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:13.655409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e8916-8582-4371-9f39-839e68465394 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:13.727787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016961bf-6db5-4b8c-9d80-4b91f0015e09 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:13.734817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:13.783045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef33c4-f60f-4a96-a7d7-680fb61fb450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:13.784240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef33c4-f60f-4a96-a7d7-680fb61fb450 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:13.784759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef33c4-f60f-4a96-a7d7-680fb61fb450 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:13.785457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef33c4-f60f-4a96-a7d7-680fb61fb450 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:13.787894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef33c4-f60f-4a96-a7d7-680fb61fb450 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:13.789560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef33c4-f60f-4a96-a7d7-680fb61fb450 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:13.791427 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 134/268] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:13 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:13.793061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea368f-996f-4bbf-8eb4-ee374b56cf66 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:13.793557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea368f-996f-4bbf-8eb4-ee374b56cf66 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:13.794209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ea368f-996f-4bbf-8eb4-ee374b56cf66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:13.815723 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b7ea368f-996f-4bbf-8eb4-ee374b56cf66 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:13.817252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b7ea368f-996f-4bbf-8eb4-ee374b56cf66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:13.880741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e8916-8582-4371-9f39-839e68465394 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:13.908817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 135/269] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:13.910028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea368f-996f-4bbf-8eb4-ee374b56cf66 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:13.926706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b205105-581a-4c61-b022-4db73b65ec55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:13.928937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b205105-581a-4c61-b022-4db73b65ec55 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:13.929426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b205105-581a-4c61-b022-4db73b65ec55 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:13.929984 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b205105-581a-4c61-b022-4db73b65ec55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:13.931861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b205105-581a-4c61-b022-4db73b65ec55 None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=87ea676d839141b4b3e827be73b4e648, user.id=38f6e23b56464e44ac078817b44e4eec) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:13.933734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b205105-581a-4c61-b022-4db73b65ec55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:13.935397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b205105-581a-4c61-b022-4db73b65ec55 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:13.948354 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 135/270] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:13 2022] GET /identity/v3/role_assignments?user.id=38f6e23b56464e44ac078817b44e4eec&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 608 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:13.949961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf292b5-5e68-47f9-8470-e70875c4a7c5 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:13.954717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf292b5-5e68-47f9-8470-e70875c4a7c5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:13.955278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf292b5-5e68-47f9-8470-e70875c4a7c5 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:13.957313 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 136/271] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:13.961548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:13.963835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1befa-4e89-4aaf-b0d7-fd89d98e28dc None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:13.964329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1befa-4e89-4aaf-b0d7-fd89d98e28dc None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:13.964798 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1befa-4e89-4aaf-b0d7-fd89d98e28dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:14.003540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-b2c1befa-4e89-4aaf-b0d7-fd89d98e28dc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:14.007302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b2c1befa-4e89-4aaf-b0d7-fd89d98e28dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:14.107480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ea368f-996f-4bbf-8eb4-ee374b56cf66 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:14.152674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812c8ef-7bfb-407d-9dbb-19c64e35b15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:14.153450 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 137/272] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:14.153896 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812c8ef-7bfb-407d-9dbb-19c64e35b15f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:14.154393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812c8ef-7bfb-407d-9dbb-19c64e35b15f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:14.154862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8812c8ef-7bfb-407d-9dbb-19c64e35b15f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:14.155830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812c8ef-7bfb-407d-9dbb-19c64e35b15f None admin] RBAC: Adding query filter params (role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf, scope.project.id=87ea676d839141b4b3e827be73b4e648, user.id=38f6e23b56464e44ac078817b44e4eec) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:14.156344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812c8ef-7bfb-407d-9dbb-19c64e35b15f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:14.158035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8812c8ef-7bfb-407d-9dbb-19c64e35b15f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:14.164724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 136/273] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:13 2022] GET /identity/v3/role_assignments?user.id=38f6e23b56464e44ac078817b44e4eec&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:14.262897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1befa-4e89-4aaf-b0d7-fd89d98e28dc None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:14.312905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:14.412950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1befa-4e89-4aaf-b0d7-fd89d98e28dc None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:14.485299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e12e7-2547-49ca-bbd4-1116b491ec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:14.485949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e12e7-2547-49ca-bbd4-1116b491ec65 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:14.486231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e12e7-2547-49ca-bbd4-1116b491ec65 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:14.486504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8e12e7-2547-49ca-bbd4-1116b491ec65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:14.499780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8e12e7-2547-49ca-bbd4-1116b491ec65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-fd8e12e7-2547-49ca-bbd4-1116b491ec65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:14.504949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:14.506339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:14.507122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 138/274] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:14.513568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812c8ef-7bfb-407d-9dbb-19c64e35b15f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:14.549709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e12e7-2547-49ca-bbd4-1116b491ec65 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:14.668226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:14.676563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8812c8ef-7bfb-407d-9dbb-19c64e35b15f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:14.681542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:14.694179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8e12e7-2547-49ca-bbd4-1116b491ec65 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:14.721352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98741f4a-4ee9-4495-8a9b-11b9daeb0540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:14.721970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741f4a-4ee9-4495-8a9b-11b9daeb0540 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:14.722250 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741f4a-4ee9-4495-8a9b-11b9daeb0540 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:14.722642 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98741f4a-4ee9-4495-8a9b-11b9daeb0540 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:14.723493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98741f4a-4ee9-4495-8a9b-11b9daeb0540 None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:14.723806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98741f4a-4ee9-4495-8a9b-11b9daeb0540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:14.726627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98741f4a-4ee9-4495-8a9b-11b9daeb0540 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:14.734320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c8fc9-7f07-458d-8469-9120bfa7c96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:14.735263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c8fc9-7f07-458d-8469-9120bfa7c96f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:14.735762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c8fc9-7f07-458d-8469-9120bfa7c96f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:14.736532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c8fc9-7f07-458d-8469-9120bfa7c96f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:14.736879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 137/275] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:14.738154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2729e-19e0-416d-96f0-dba904218913 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:14.738440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2729e-19e0-416d-96f0-dba904218913 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:14.738709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c2729e-19e0-416d-96f0-dba904218913 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:14.739113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c8fc9-7f07-458d-8469-9120bfa7c96f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:14.740828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 138/276] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:14.741830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c8fc9-7f07-458d-8469-9120bfa7c96f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:14.743211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2729e-19e0-416d-96f0-dba904218913 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:14.744601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 139/277] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:14.745968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5997c95-6f05-42bc-9bfa-a5ca2569cbfc None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:14.746252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5997c95-6f05-42bc-9bfa-a5ca2569cbfc None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:14.746524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5997c95-6f05-42bc-9bfa-a5ca2569cbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:14.765792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-f5997c95-6f05-42bc-9bfa-a5ca2569cbfc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:14.766926 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f5997c95-6f05-42bc-9bfa-a5ca2569cbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:14.845996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 140/278] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:14.848833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5997c95-6f05-42bc-9bfa-a5ca2569cbfc None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:14.863397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c2729e-19e0-416d-96f0-dba904218913 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:14.907364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104813f9-873f-476f-b317-374cf464da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:14.908019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104813f9-873f-476f-b317-374cf464da5d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:14.908295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104813f9-873f-476f-b317-374cf464da5d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:14.908564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104813f9-873f-476f-b317-374cf464da5d None admin] PATH_INFO: `/v3/users/76cf184ce26c40678c4957d8ef06d1b3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:14.922382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104813f9-873f-476f-b317-374cf464da5d None admin] RBAC: Authorizing `identity:get_user(user_id=76cf184ce26c40678c4957d8ef06d1b3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:14.923867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104813f9-873f-476f-b317-374cf464da5d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:14.925786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 139/279] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:14 2022] GET /identity/v3/users/76cf184ce26c40678c4957d8ef06d1b3 => generated 293 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:14.926693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e7c12-bad5-443e-8857-b9f3319e926a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:14.927517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e7c12-bad5-443e-8857-b9f3319e926a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:14.927968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8e7c12-bad5-443e-8857-b9f3319e926a None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:14.933157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 140/280] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:14.933157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e7c12-bad5-443e-8857-b9f3319e926a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:14.948046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:14.966551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:14.981832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5997c95-6f05-42bc-9bfa-a5ca2569cbfc None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:15.017892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e736037-b7d2-4415-9726-722cd8768381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:15.018418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e736037-b7d2-4415-9726-722cd8768381 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:15.018693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e736037-b7d2-4415-9726-722cd8768381 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:15.018963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e736037-b7d2-4415-9726-722cd8768381 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:15.019634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e736037-b7d2-4415-9726-722cd8768381 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:15.021185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e736037-b7d2-4415-9726-722cd8768381 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:15.043559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8e7c12-bad5-443e-8857-b9f3319e926a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:15.050247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 141/281] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:00:14 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:00:15.051933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b62d83-db76-421d-9896-5771f5ea3850 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:15.052974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b62d83-db76-421d-9896-5771f5ea3850 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:15.053282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b62d83-db76-421d-9896-5771f5ea3850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:15.075650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c3b62d83-db76-421d-9896-5771f5ea3850 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:15.077088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c3b62d83-db76-421d-9896-5771f5ea3850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:15.097663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e9951-6ebf-46b7-928e-2a3728771b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:15.098388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e9951-6ebf-46b7-928e-2a3728771b26 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:15.098735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e9951-6ebf-46b7-928e-2a3728771b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:15.099169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145e9951-6ebf-46b7-928e-2a3728771b26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:15.100363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e9951-6ebf-46b7-928e-2a3728771b26 None admin] RBAC: Adding query filter params (role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, user.id=76cf184ce26c40678c4957d8ef06d1b3, scope.system=all) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:15.100755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e9951-6ebf-46b7-928e-2a3728771b26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:15.102802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145e9951-6ebf-46b7-928e-2a3728771b26 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:15.110773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 141/282] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Sep 10 14:00:14 2022] GET /identity/v3/role_assignments?user.id=76cf184ce26c40678c4957d8ef06d1b3&scope.system=all&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:15.119853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e9951-6ebf-46b7-928e-2a3728771b26 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:15.167479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 142/283] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:15.228616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:15.257186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145e9951-6ebf-46b7-928e-2a3728771b26 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:15.302640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0780c6c-879f-4b49-8eaa-c725e5449ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:15.303209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0780c6c-879f-4b49-8eaa-c725e5449ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:15.303513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0780c6c-879f-4b49-8eaa-c725e5449ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:15.303774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0780c6c-879f-4b49-8eaa-c725e5449ee1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:15.314671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0780c6c-879f-4b49-8eaa-c725e5449ee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-e0780c6c-879f-4b49-8eaa-c725e5449ee1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:15.320223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:15.321102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:15.321878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 142/284] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:00:15.327535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b62d83-db76-421d-9896-5771f5ea3850 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:15.406068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:15.444972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b62d83-db76-421d-9896-5771f5ea3850 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:15.454258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0780c6c-879f-4b49-8eaa-c725e5449ee1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:15.491047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da7d7f-bbc6-4cec-89db-7380be5fed00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:15.491703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7d7f-bbc6-4cec-89db-7380be5fed00 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:15.491986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7d7f-bbc6-4cec-89db-7380be5fed00 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:15.492258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7d7f-bbc6-4cec-89db-7380be5fed00 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:15.492938 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da7d7f-bbc6-4cec-89db-7380be5fed00 None admin] RBAC: Adding query filter params (name=member) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:15.493244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da7d7f-bbc6-4cec-89db-7380be5fed00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:15.494651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da7d7f-bbc6-4cec-89db-7380be5fed00 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:15.504618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 143/285] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:15.510749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da7d7f-bbc6-4cec-89db-7380be5fed00 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:15.582952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0780c6c-879f-4b49-8eaa-c725e5449ee1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:15.631176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10a4af-1c7d-4599-a333-31f5c09cbf5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:15.632182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10a4af-1c7d-4599-a333-31f5c09cbf5b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:15.632684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10a4af-1c7d-4599-a333-31f5c09cbf5b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:15.633158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10a4af-1c7d-4599-a333-31f5c09cbf5b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:15.633903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da7d7f-bbc6-4cec-89db-7380be5fed00 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:15.641110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:15.647703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10a4af-1c7d-4599-a333-31f5c09cbf5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9c10a4af-1c7d-4599-a333-31f5c09cbf5b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:15.652827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:15.653662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:15.654438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:15.657013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 143/286] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:15.663130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10a4af-1c7d-4599-a333-31f5c09cbf5b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:15.673472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1082a-3e36-4590-a551-1af35d8b5c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:15.674108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1082a-3e36-4590-a551-1af35d8b5c59 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:15.674389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1082a-3e36-4590-a551-1af35d8b5c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:15.674664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f1082a-3e36-4590-a551-1af35d8b5c59 None admin] PATH_INFO: `/v3/users/8b3701c605214ba5854ab5e003f9bd17` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:15.676034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1082a-3e36-4590-a551-1af35d8b5c59 None admin] RBAC: Authorizing `identity:get_user(user_id=8b3701c605214ba5854ab5e003f9bd17)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:15.678581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f1082a-3e36-4590-a551-1af35d8b5c59 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:15.680816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 144/287] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:15 2022] GET /identity/v3/users/8b3701c605214ba5854ab5e003f9bd17 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:15.686682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1082a-3e36-4590-a551-1af35d8b5c59 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:15.691090 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:15.780288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10a4af-1c7d-4599-a333-31f5c09cbf5b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:15.816076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f1082a-3e36-4590-a551-1af35d8b5c59 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:15.818861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e89ca-1466-4330-aef5-aacf73e7f66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:15.819925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e89ca-1466-4330-aef5-aacf73e7f66d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:15.820409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e89ca-1466-4330-aef5-aacf73e7f66d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:15.820880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4e89ca-1466-4330-aef5-aacf73e7f66d None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:15.821760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e89ca-1466-4330-aef5-aacf73e7f66d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:15.822275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e89ca-1466-4330-aef5-aacf73e7f66d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:15.823987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4e89ca-1466-4330-aef5-aacf73e7f66d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:15.832905 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 144/288] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:15.839688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e89ca-1466-4330-aef5-aacf73e7f66d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:15.866082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3afd39-3820-4766-a2f7-eea3e3657403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:15.866842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3afd39-3820-4766-a2f7-eea3e3657403 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:15.867314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3afd39-3820-4766-a2f7-eea3e3657403 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:15.867649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3afd39-3820-4766-a2f7-eea3e3657403 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:15.940119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:15.990562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3afd39-3820-4766-a2f7-eea3e3657403 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:15.992309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3afd39-3820-4766-a2f7-eea3e3657403 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:15.995958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:16.001284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4e89ca-1466-4330-aef5-aacf73e7f66d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:16.052268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69409f8-6b05-44b7-afef-2212a138719c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:16.055564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69409f8-6b05-44b7-afef-2212a138719c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:16.055863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69409f8-6b05-44b7-afef-2212a138719c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:16.056140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69409f8-6b05-44b7-afef-2212a138719c None admin] PATH_INFO: `/v3/users/99eea39b5aeb47d68a5d3cdf6d283861` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:16.059975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69409f8-6b05-44b7-afef-2212a138719c None admin] RBAC: Authorizing `identity:get_user(user_id=99eea39b5aeb47d68a5d3cdf6d283861)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:16.061468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69409f8-6b05-44b7-afef-2212a138719c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:16.063250 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 145/289] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:15 2022] GET /identity/v3/users/99eea39b5aeb47d68a5d3cdf6d283861 => generated 297 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:16.069573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69409f8-6b05-44b7-afef-2212a138719c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:16.180923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 145/290] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:16.203010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3afd39-3820-4766-a2f7-eea3e3657403 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:16.318110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69409f8-6b05-44b7-afef-2212a138719c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:16.322571 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3afd39-3820-4766-a2f7-eea3e3657403 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:16.326075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:16.369218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9c4d-0dc8-47a4-89b7-e5363559c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:16.370256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9c4d-0dc8-47a4-89b7-e5363559c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:16.371950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9c4d-0dc8-47a4-89b7-e5363559c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:16.372375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed9c4d-0dc8-47a4-89b7-e5363559c1a5 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:16.375669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed9c4d-0dc8-47a4-89b7-e5363559c1a5 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:16.377307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed9c4d-0dc8-47a4-89b7-e5363559c1a5 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:16.382296 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 146/291] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:16 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:16.391485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:16.399760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9c4d-0dc8-47a4-89b7-e5363559c1a5 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:16.411958 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b183a7-786f-4144-a189-7ecf8a0f037b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:16.412797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b183a7-786f-4144-a189-7ecf8a0f037b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:16.413075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b183a7-786f-4144-a189-7ecf8a0f037b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:16.413347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b183a7-786f-4144-a189-7ecf8a0f037b None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:16.414566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b183a7-786f-4144-a189-7ecf8a0f037b None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:16.416065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b183a7-786f-4144-a189-7ecf8a0f037b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:16.417987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 146/292] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:16 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:16.425472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b183a7-786f-4144-a189-7ecf8a0f037b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:16.571351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b183a7-786f-4144-a189-7ecf8a0f037b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:16.585035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed9c4d-0dc8-47a4-89b7-e5363559c1a5 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:16.642700 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0898b780-952d-45f4-8039-b8c7f907fb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:16.643310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b780-952d-45f4-8039-b8c7f907fb7a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:16.646165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b780-952d-45f4-8039-b8c7f907fb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:16.646165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0898b780-952d-45f4-8039-b8c7f907fb7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:16.646337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d8d238-4907-4256-9883-04ad118e7cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:16.646423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898b780-952d-45f4-8039-b8c7f907fb7a None admin] RBAC: Adding query filter params (role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, scope.project.id=87ea676d839141b4b3e827be73b4e648, user.id=8b3701c605214ba5854ab5e003f9bd17) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:16.646523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8d238-4907-4256-9883-04ad118e7cfe None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:16.646594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898b780-952d-45f4-8039-b8c7f907fb7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:16.646660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8d238-4907-4256-9883-04ad118e7cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:16.646660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d8d238-4907-4256-9883-04ad118e7cfe None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648/users/99eea39b5aeb47d68a5d3cdf6d283861/roles/efc3726869ca4763809656e2e7e1646a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:16.646775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0898b780-952d-45f4-8039-b8c7f907fb7a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:16.652200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8d238-4907-4256-9883-04ad118e7cfe None admin] RBAC: Authorizing `identity:create_grant(project_id=87ea676d839141b4b3e827be73b4e648, user_id=99eea39b5aeb47d68a5d3cdf6d283861, role_id=efc3726869ca4763809656e2e7e1646a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:16.652723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 147/293] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:16 2022] GET /identity/v3/role_assignments?user.id=8b3701c605214ba5854ab5e003f9bd17&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:16.654089 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d8d238-4907-4256-9883-04ad118e7cfe None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:16.686079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 147/294] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:00:16 2022] PUT /identity/v3/projects/87ea676d839141b4b3e827be73b4e648/users/99eea39b5aeb47d68a5d3cdf6d283861/roles/efc3726869ca4763809656e2e7e1646a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:00:17.558237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dff28-29cd-4e51-ab0e-283377b108bf None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:17.558783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dff28-29cd-4e51-ab0e-283377b108bf None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:17.559050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dff28-29cd-4e51-ab0e-283377b108bf None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:17.560541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 148/295] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:17.566040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6b332-f5dd-436b-b883-27197451d3f9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:17.566469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6b332-f5dd-436b-b883-27197451d3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:17.566766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6b332-f5dd-436b-b883-27197451d3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:17.585497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e2b6b332-f5dd-436b-b883-27197451d3f9 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:17.586693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e2b6b332-f5dd-436b-b883-27197451d3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:17.642268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:17.665324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 148/296] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:17.756756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8b7a7-2729-455a-8ea7-012278754ce2 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:17.757190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8b7a7-2729-455a-8ea7-012278754ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:17.757685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8b7a7-2729-455a-8ea7-012278754ce2 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:17.759969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 149/297] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:17.766152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916ac69-c2a6-4f65-a37b-f3cab8ef5ba6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:17.766835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916ac69-c2a6-4f65-a37b-f3cab8ef5ba6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:17.767321 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916ac69-c2a6-4f65-a37b-f3cab8ef5ba6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:17.786096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f916ac69-c2a6-4f65-a37b-f3cab8ef5ba6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:17.787939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f916ac69-c2a6-4f65-a37b-f3cab8ef5ba6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:17.860447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 149/298] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:17.948737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8b7a7-2729-455a-8ea7-012278754ce2 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:17.963303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8198a-f29f-4e67-9548-04905b605581 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:17.963605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8198a-f29f-4e67-9548-04905b605581 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:17.963876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c8198a-f29f-4e67-9548-04905b605581 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:17.965213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 150/299] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:00:17.970389 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df975-cac3-4b5f-a872-63b4d486b512 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:17.970670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df975-cac3-4b5f-a872-63b4d486b512 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:17.970948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282df975-cac3-4b5f-a872-63b4d486b512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:17.990513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-282df975-cac3-4b5f-a872-63b4d486b512 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:17.991716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-282df975-cac3-4b5f-a872-63b4d486b512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:17.992729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:18.068407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8b7a7-2729-455a-8ea7-012278754ce2 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:18.083820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 151/300] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:18.114108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03b4bb-2150-47d4-a19f-73213ffc2127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:18.114952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03b4bb-2150-47d4-a19f-73213ffc2127 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:18.115522 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03b4bb-2150-47d4-a19f-73213ffc2127 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:18.115801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03b4bb-2150-47d4-a19f-73213ffc2127 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:18.126575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a03b4bb-2150-47d4-a19f-73213ffc2127 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-6a03b4bb-2150-47d4-a19f-73213ffc2127 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:18.132013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:18.132865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:18.133620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 150/301] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:18.139050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282df975-cac3-4b5f-a872-63b4d486b512 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:18.143462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03b4bb-2150-47d4-a19f-73213ffc2127 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:18.193080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:18.253207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282df975-cac3-4b5f-a872-63b4d486b512 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:18.254725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:18.263663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a03b4bb-2150-47d4-a19f-73213ffc2127 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:18.292435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa233e7-0efa-414b-af1f-fd01a25f26ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:18.293044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa233e7-0efa-414b-af1f-fd01a25f26ce None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:18.293323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa233e7-0efa-414b-af1f-fd01a25f26ce None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:18.293593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa233e7-0efa-414b-af1f-fd01a25f26ce None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:18.294258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa233e7-0efa-414b-af1f-fd01a25f26ce None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:18.294559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa233e7-0efa-414b-af1f-fd01a25f26ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:18.295988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa233e7-0efa-414b-af1f-fd01a25f26ce None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:18.304525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 152/302] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:18.305881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5412850-aabd-434d-867c-1ad4fb5a56bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:18.306515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5412850-aabd-434d-867c-1ad4fb5a56bb None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:18.306796 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5412850-aabd-434d-867c-1ad4fb5a56bb None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:18.307065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5412850-aabd-434d-867c-1ad4fb5a56bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:18.310005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa233e7-0efa-414b-af1f-fd01a25f26ce None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:18.318818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5412850-aabd-434d-867c-1ad4fb5a56bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-a5412850-aabd-434d-867c-1ad4fb5a56bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:18.324366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:18.326205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:18.327189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 151/303] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:18.332657 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5412850-aabd-434d-867c-1ad4fb5a56bb None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:18.425694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa233e7-0efa-414b-af1f-fd01a25f26ce None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:18.437170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:18.453571 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5412850-aabd-434d-867c-1ad4fb5a56bb None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:18.467530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088809cb-5ba5-4e30-b087-888282578003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:18.468165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088809cb-5ba5-4e30-b087-888282578003 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:18.468443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088809cb-5ba5-4e30-b087-888282578003 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:18.468717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088809cb-5ba5-4e30-b087-888282578003 None admin] PATH_INFO: `/v3/users/76cf184ce26c40678c4957d8ef06d1b3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:18.470105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088809cb-5ba5-4e30-b087-888282578003 None admin] RBAC: Authorizing `identity:get_user(user_id=76cf184ce26c40678c4957d8ef06d1b3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:18.471579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088809cb-5ba5-4e30-b087-888282578003 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:18.473161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 153/304] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:18 2022] GET /identity/v3/users/76cf184ce26c40678c4957d8ef06d1b3 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:18.476399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088809cb-5ba5-4e30-b087-888282578003 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:18.491039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10d57c-ce5f-4c21-a925-a68ab9386895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:18.491823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10d57c-ce5f-4c21-a925-a68ab9386895 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:18.492100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10d57c-ce5f-4c21-a925-a68ab9386895 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:18.492365 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb10d57c-ce5f-4c21-a925-a68ab9386895 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:18.493019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10d57c-ce5f-4c21-a925-a68ab9386895 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:18.493318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10d57c-ce5f-4c21-a925-a68ab9386895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:18.494702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb10d57c-ce5f-4c21-a925-a68ab9386895 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:18.504195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 152/305] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:18.504967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10d57c-ce5f-4c21-a925-a68ab9386895 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:18.525632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:18.588863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088809cb-5ba5-4e30-b087-888282578003 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:18.612997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb10d57c-ce5f-4c21-a925-a68ab9386895 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:18.623460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6141f-cd1e-4591-bd03-9d8c74a5ef1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:18.624091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6141f-cd1e-4591-bd03-9d8c74a5ef1f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:18.624370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6141f-cd1e-4591-bd03-9d8c74a5ef1f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:18.624640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a6141f-cd1e-4591-bd03-9d8c74a5ef1f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:18.625802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6141f-cd1e-4591-bd03-9d8c74a5ef1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:18.627283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a6141f-cd1e-4591-bd03-9d8c74a5ef1f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:18.631494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 154/306] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:18.631494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6141f-cd1e-4591-bd03-9d8c74a5ef1f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:18.655604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65b924-ec76-44dd-8462-8401c7f13c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:18.656253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b924-ec76-44dd-8462-8401c7f13c94 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:18.656529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b924-ec76-44dd-8462-8401c7f13c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:18.656799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba65b924-ec76-44dd-8462-8401c7f13c94 None admin] PATH_INFO: `/v3/system/users/76cf184ce26c40678c4957d8ef06d1b3/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:18.658263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65b924-ec76-44dd-8462-8401c7f13c94 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=76cf184ce26c40678c4957d8ef06d1b3, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:18.659689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba65b924-ec76-44dd-8462-8401c7f13c94 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:18.676585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 153/307] 10.222.0.26 () {58 vars in 1334 bytes} [Sat Sep 10 14:00:18 2022] PUT /identity/v3/system/users/76cf184ce26c40678c4957d8ef06d1b3/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:18.676585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65b924-ec76-44dd-8462-8401c7f13c94 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:18.680489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:18.779177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a6141f-cd1e-4591-bd03-9d8c74a5ef1f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:18.782224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:18.809999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba65b924-ec76-44dd-8462-8401c7f13c94 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:18.839323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80bf0e-ebff-460a-8337-f2ed22184ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:18.840269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80bf0e-ebff-460a-8337-f2ed22184ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:18.840611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80bf0e-ebff-460a-8337-f2ed22184ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:18.840950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80bf0e-ebff-460a-8337-f2ed22184ff4 None admin] PATH_INFO: `/v3/users/23f85921e67542a3a37b711d3ace53f8` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:18.850540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac877915-9da8-4cdb-83df-c91c72727ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:18.851254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac877915-9da8-4cdb-83df-c91c72727ec6 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:18.851652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac877915-9da8-4cdb-83df-c91c72727ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:18.851998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac877915-9da8-4cdb-83df-c91c72727ec6 None admin] PATH_INFO: `/v3/groups` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:18.852636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac877915-9da8-4cdb-83df-c91c72727ec6 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:18.854095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac877915-9da8-4cdb-83df-c91c72727ec6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:18.860201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80bf0e-ebff-460a-8337-f2ed22184ff4 None admin] RBAC: Authorizing `identity:get_user(user_id=23f85921e67542a3a37b711d3ace53f8)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:18.861167 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c80bf0e-ebff-460a-8337-f2ed22184ff4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:18.863598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 155/308] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:18 2022] GET /identity/v3/users/23f85921e67542a3a37b711d3ace53f8 => generated 293 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:18.870504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80bf0e-ebff-460a-8337-f2ed22184ff4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:18.877893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 154/309] 10.222.0.26 () {60 vars in 1134 bytes} [Sat Sep 10 14:00:18 2022] POST /identity/v3/groups => generated 229 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:00:18.998769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80bf0e-ebff-460a-8337-f2ed22184ff4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:19.001097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:19.043880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fdbf8-4cd5-48e0-9f64-9518be5efa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:19.044494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fdbf8-4cd5-48e0-9f64-9518be5efa51 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:19.044768 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fdbf8-4cd5-48e0-9f64-9518be5efa51 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:19.045040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fdbf8-4cd5-48e0-9f64-9518be5efa51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:19.045784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fdbf8-4cd5-48e0-9f64-9518be5efa51 None admin] RBAC: Adding query filter params (role.id=efc3726869ca4763809656e2e7e1646a, user.id=23f85921e67542a3a37b711d3ace53f8, scope.system=all) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:19.046087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fdbf8-4cd5-48e0-9f64-9518be5efa51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:19.047593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fdbf8-4cd5-48e0-9f64-9518be5efa51 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:19.059595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 156/310] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Sep 10 14:00:18 2022] GET /identity/v3/role_assignments?user.id=23f85921e67542a3a37b711d3ace53f8&scope.system=all&role.id=efc3726869ca4763809656e2e7e1646a => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:19.104446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f195f30-c6e0-4881-9d6a-46fa6d2ae265 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:19.104737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f195f30-c6e0-4881-9d6a-46fa6d2ae265 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:19.104919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f195f30-c6e0-4881-9d6a-46fa6d2ae265 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:19.106279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 155/311] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:19.112262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be689a95-b138-45a0-a9f4-3fb60a27e954 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:19.112577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be689a95-b138-45a0-a9f4-3fb60a27e954 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:19.112808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be689a95-b138-45a0-a9f4-3fb60a27e954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:19.132714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-be689a95-b138-45a0-a9f4-3fb60a27e954 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:19.133960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-be689a95-b138-45a0-a9f4-3fb60a27e954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:19.212741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 157/312] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:19.485089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f195f30-c6e0-4881-9d6a-46fa6d2ae265 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:19.588803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f195f30-c6e0-4881-9d6a-46fa6d2ae265 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:19.623751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4cbaf-e453-4383-85e9-6fb04b1ef229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:19.624404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4cbaf-e453-4383-85e9-6fb04b1ef229 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:19.624641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4cbaf-e453-4383-85e9-6fb04b1ef229 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:19.624911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4cbaf-e453-4383-85e9-6fb04b1ef229 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:19.635370 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4cbaf-e453-4383-85e9-6fb04b1ef229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-cef4cbaf-e453-4383-85e9-6fb04b1ef229 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:19.640552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:19.641443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:19.642439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 156/313] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:19.648404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be689a95-b138-45a0-a9f4-3fb60a27e954 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:19.757200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be689a95-b138-45a0-a9f4-3fb60a27e954 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:19.758991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:19.797839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585da56-7e44-452c-8a01-8be2d2459e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:19.798455 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585da56-7e44-452c-8a01-8be2d2459e4e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:19.798728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585da56-7e44-452c-8a01-8be2d2459e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:19.798996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e585da56-7e44-452c-8a01-8be2d2459e4e None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:19.799690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585da56-7e44-452c-8a01-8be2d2459e4e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:19.800041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585da56-7e44-452c-8a01-8be2d2459e4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:19.801432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e585da56-7e44-452c-8a01-8be2d2459e4e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:19.811039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 158/314] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:19.816711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4cbaf-e453-4383-85e9-6fb04b1ef229 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:19.821075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:19.928865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4cbaf-e453-4383-85e9-6fb04b1ef229 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:19.969272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f000d7-549d-436e-9086-562b4ea102c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:19.969916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f000d7-549d-436e-9086-562b4ea102c6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:19.970191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f000d7-549d-436e-9086-562b4ea102c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:19.970460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f000d7-549d-436e-9086-562b4ea102c6 None admin] PATH_INFO: `/v3/users/99eea39b5aeb47d68a5d3cdf6d283861` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:19.971620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f000d7-549d-436e-9086-562b4ea102c6 None admin] RBAC: Authorizing `identity:get_user(user_id=99eea39b5aeb47d68a5d3cdf6d283861)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:19.973066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f000d7-549d-436e-9086-562b4ea102c6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:19.975685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 157/315] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:19 2022] GET /identity/v3/users/99eea39b5aeb47d68a5d3cdf6d283861 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:19.981361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585da56-7e44-452c-8a01-8be2d2459e4e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:20.089565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e585da56-7e44-452c-8a01-8be2d2459e4e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:20.124605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bf66d-2d3e-46e9-9b7d-1a4dae198c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:20.125206 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bf66d-2d3e-46e9-9b7d-1a4dae198c8b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:20.125480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bf66d-2d3e-46e9-9b7d-1a4dae198c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:20.125750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6bf66d-2d3e-46e9-9b7d-1a4dae198c8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:20.153549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:20.196112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bf66d-2d3e-46e9-9b7d-1a4dae198c8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:20.197572 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6bf66d-2d3e-46e9-9b7d-1a4dae198c8b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:20.304004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 159/316] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:20.309335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f000d7-549d-436e-9086-562b4ea102c6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:20.385704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:20.419208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f000d7-549d-436e-9086-562b4ea102c6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:20.454749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5154eb88-3850-44e3-966b-0877438f1ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:20.455382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154eb88-3850-44e3-966b-0877438f1ccf None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:20.455677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154eb88-3850-44e3-966b-0877438f1ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:20.455948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5154eb88-3850-44e3-966b-0877438f1ccf None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:20.457048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154eb88-3850-44e3-966b-0877438f1ccf None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:20.459164 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5154eb88-3850-44e3-966b-0877438f1ccf None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:20.461107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 158/317] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:20 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:20.466866 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bf66d-2d3e-46e9-9b7d-1a4dae198c8b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:20.527212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:20.579734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6bf66d-2d3e-46e9-9b7d-1a4dae198c8b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:20.614321 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81694f-2397-45d3-bf90-9f38d8081ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:20.615409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81694f-2397-45d3-bf90-9f38d8081ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:20.616304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81694f-2397-45d3-bf90-9f38d8081ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:20.616779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81694f-2397-45d3-bf90-9f38d8081ae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:20.617704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81694f-2397-45d3-bf90-9f38d8081ae9 None admin] RBAC: Adding query filter params (role.id=efc3726869ca4763809656e2e7e1646a, scope.project.id=87ea676d839141b4b3e827be73b4e648, user.id=99eea39b5aeb47d68a5d3cdf6d283861) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:20.618300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81694f-2397-45d3-bf90-9f38d8081ae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:20.619981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81694f-2397-45d3-bf90-9f38d8081ae9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:20.626284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 160/318] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:20 2022] GET /identity/v3/role_assignments?user.id=99eea39b5aeb47d68a5d3cdf6d283861&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=efc3726869ca4763809656e2e7e1646a => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:20.989277 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c351ec2-6acb-45f2-9736-03804b6d3345 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:20.990404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c351ec2-6acb-45f2-9736-03804b6d3345 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:20.990569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c351ec2-6acb-45f2-9736-03804b6d3345 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:20.992149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 159/319] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:20.997390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ced921-24ef-4df7-ad9c-5fb0028cc3d4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:20.997796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ced921-24ef-4df7-ad9c-5fb0028cc3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:20.997956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ced921-24ef-4df7-ad9c-5fb0028cc3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:21.016540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-73ced921-24ef-4df7-ad9c-5fb0028cc3d4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:21.017707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-73ced921-24ef-4df7-ad9c-5fb0028cc3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:21.089681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 161/320] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:21.195797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169cf76-04c2-4ea6-9ded-2563f6dc6b2f None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:21.196081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169cf76-04c2-4ea6-9ded-2563f6dc6b2f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:21.196514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4169cf76-04c2-4ea6-9ded-2563f6dc6b2f None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:21.198023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 160/321] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:21.203339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38ce3c-0dd0-4b16-87b9-9ad14b87041b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:21.203658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38ce3c-0dd0-4b16-87b9-9ad14b87041b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:21.203927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38ce3c-0dd0-4b16-87b9-9ad14b87041b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:21.222575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fd38ce3c-0dd0-4b16-87b9-9ad14b87041b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:21.224405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fd38ce3c-0dd0-4b16-87b9-9ad14b87041b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:21.273641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:21.294923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 162/322] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:21.365917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169cf76-04c2-4ea6-9ded-2563f6dc6b2f None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:21.413586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f731-f92f-41c2-be50-31c33f326eb6 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:21.413863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f731-f92f-41c2-be50-31c33f326eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:21.414340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9f731-f92f-41c2-be50-31c33f326eb6 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:21.415994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 163/323] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:21.420505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876fd68-321c-437a-a4bc-36ac784289a0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:21.420941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876fd68-321c-437a-a4bc-36ac784289a0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:21.421215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876fd68-321c-437a-a4bc-36ac784289a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:21.440610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9876fd68-321c-437a-a4bc-36ac784289a0 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:21.441789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9876fd68-321c-437a-a4bc-36ac784289a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:21.489752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4169cf76-04c2-4ea6-9ded-2563f6dc6b2f None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:21.513440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 164/324] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:21.527623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d2932-243c-4eef-b759-bf70efb00b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:21.528191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d2932-243c-4eef-b759-bf70efb00b2b None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:21.528465 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d2932-243c-4eef-b759-bf70efb00b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:21.528734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d2932-243c-4eef-b759-bf70efb00b2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:21.535723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:21.540988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d2932-243c-4eef-b759-bf70efb00b2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-952d2932-243c-4eef-b759-bf70efb00b2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:21.546149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:21.546986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:21.552983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 161/325] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:21.554452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876fd68-321c-437a-a4bc-36ac784289a0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:21.567881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d2932-243c-4eef-b759-bf70efb00b2b None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:21.667127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876fd68-321c-437a-a4bc-36ac784289a0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:21.671267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d2932-243c-4eef-b759-bf70efb00b2b None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:21.682457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:21.705234 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04514874-38b1-4eeb-8abf-ff77dc9d3b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:21.705836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04514874-38b1-4eeb-8abf-ff77dc9d3b0d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:21.706111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04514874-38b1-4eeb-8abf-ff77dc9d3b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:21.706380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04514874-38b1-4eeb-8abf-ff77dc9d3b0d None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:21.707033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04514874-38b1-4eeb-8abf-ff77dc9d3b0d None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:21.707330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04514874-38b1-4eeb-8abf-ff77dc9d3b0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:21.707814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eeff2-bc16-4c39-8b30-5e046003e074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:21.708964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eeff2-bc16-4c39-8b30-5e046003e074 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:21.709240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eeff2-bc16-4c39-8b30-5e046003e074 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:21.709331 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04514874-38b1-4eeb-8abf-ff77dc9d3b0d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:21.709509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eeff2-bc16-4c39-8b30-5e046003e074 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:21.710612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866eeff2-bc16-4c39-8b30-5e046003e074 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:21.712084 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866eeff2-bc16-4c39-8b30-5e046003e074 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:21.713815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 162/326] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:21.717792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 165/327] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:21.720976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eeff2-bc16-4c39-8b30-5e046003e074 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:21.724582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04514874-38b1-4eeb-8abf-ff77dc9d3b0d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:21.793697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:21.825839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04514874-38b1-4eeb-8abf-ff77dc9d3b0d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:21.828033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eeff2-bc16-4c39-8b30-5e046003e074 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:21.859826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9a277-d8cb-4b0d-a3df-744d9c353e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:21.860446 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9a277-d8cb-4b0d-a3df-744d9c353e9f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:21.860717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9a277-d8cb-4b0d-a3df-744d9c353e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:21.860987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b9a277-d8cb-4b0d-a3df-744d9c353e9f None admin] PATH_INFO: `/v3/users/76cf184ce26c40678c4957d8ef06d1b3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:21.861455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20de7dc-02e1-4e7d-8678-95833181cc90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:21.862050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20de7dc-02e1-4e7d-8678-95833181cc90 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:21.862138 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9a277-d8cb-4b0d-a3df-744d9c353e9f None admin] RBAC: Authorizing `identity:get_user(user_id=76cf184ce26c40678c4957d8ef06d1b3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:21.862332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20de7dc-02e1-4e7d-8678-95833181cc90 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:21.862588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20de7dc-02e1-4e7d-8678-95833181cc90 None admin] PATH_INFO: `/v3/groups` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:21.863202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20de7dc-02e1-4e7d-8678-95833181cc90 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:21.863532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b9a277-d8cb-4b0d-a3df-744d9c353e9f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:21.865369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20de7dc-02e1-4e7d-8678-95833181cc90 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:21.865655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 166/328] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:21 2022] GET /identity/v3/users/76cf184ce26c40678c4957d8ef06d1b3 => generated 293 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:21.866374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9a277-d8cb-4b0d-a3df-744d9c353e9f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:21.880147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 163/329] 10.222.0.26 () {60 vars in 1134 bytes} [Sat Sep 10 14:00:21 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:00:21.884477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20de7dc-02e1-4e7d-8678-95833181cc90 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:21.914998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:21.981285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b9a277-d8cb-4b0d-a3df-744d9c353e9f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:22.004827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20de7dc-02e1-4e7d-8678-95833181cc90 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:22.019793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee8a50-e0fa-4245-b723-a3e289fe2971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:22.020444 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee8a50-e0fa-4245-b723-a3e289fe2971 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:22.020798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee8a50-e0fa-4245-b723-a3e289fe2971 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:22.021170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee8a50-e0fa-4245-b723-a3e289fe2971 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:22.033231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee8a50-e0fa-4245-b723-a3e289fe2971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:22.035774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-56ee8a50-e0fa-4245-b723-a3e289fe2971 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:22.040443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:22.047556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:22.048352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 167/330] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:22.049480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee8a50-e0fa-4245-b723-a3e289fe2971 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:22.054185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cb761-5ea9-4195-acce-a46b7cbf7927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:22.054801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cb761-5ea9-4195-acce-a46b7cbf7927 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:22.055474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cb761-5ea9-4195-acce-a46b7cbf7927 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:22.055474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585cb761-5ea9-4195-acce-a46b7cbf7927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:22.056096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cb761-5ea9-4195-acce-a46b7cbf7927 None admin] RBAC: Adding query filter params (role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, user.id=76cf184ce26c40678c4957d8ef06d1b3, scope.system=all) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:22.056397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cb761-5ea9-4195-acce-a46b7cbf7927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:22.057840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585cb761-5ea9-4195-acce-a46b7cbf7927 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:22.072254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 164/331] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Sep 10 14:00:21 2022] GET /identity/v3/role_assignments?user.id=76cf184ce26c40678c4957d8ef06d1b3&scope.system=all&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 510 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:22.169734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee8a50-e0fa-4245-b723-a3e289fe2971 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:22.171466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:22.211739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb686e9-1517-4ee3-9c40-790e13e09d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:22.212717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb686e9-1517-4ee3-9c40-790e13e09d5e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:22.213192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb686e9-1517-4ee3-9c40-790e13e09d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:22.213647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb686e9-1517-4ee3-9c40-790e13e09d5e None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:22.214406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb686e9-1517-4ee3-9c40-790e13e09d5e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:22.214652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb686e9-1517-4ee3-9c40-790e13e09d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:22.216141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb686e9-1517-4ee3-9c40-790e13e09d5e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:22.226793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 168/332] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:22.232508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cb761-5ea9-4195-acce-a46b7cbf7927 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:22.348445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585cb761-5ea9-4195-acce-a46b7cbf7927 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:22.385499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ba7f5-abfa-4ffc-9014-38c611240f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:22.386431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ba7f5-abfa-4ffc-9014-38c611240f8a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:22.386911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ba7f5-abfa-4ffc-9014-38c611240f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:22.387184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ba7f5-abfa-4ffc-9014-38c611240f8a None admin] PATH_INFO: `/v3/users/23f85921e67542a3a37b711d3ace53f8` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:22.388351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ba7f5-abfa-4ffc-9014-38c611240f8a None admin] RBAC: Authorizing `identity:get_user(user_id=23f85921e67542a3a37b711d3ace53f8)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:22.389780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27ba7f5-abfa-4ffc-9014-38c611240f8a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:22.391547 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 165/333] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:22 2022] GET /identity/v3/users/23f85921e67542a3a37b711d3ace53f8 => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:22.397149 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb686e9-1517-4ee3-9c40-790e13e09d5e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:22.515232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb686e9-1517-4ee3-9c40-790e13e09d5e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:22.552672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2df2e-2a43-43b5-90a1-ee8ec8b93f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:22.553389 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2df2e-2a43-43b5-90a1-ee8ec8b93f9d None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:22.553654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2df2e-2a43-43b5-90a1-ee8ec8b93f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:22.553917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2df2e-2a43-43b5-90a1-ee8ec8b93f9d None admin] PATH_INFO: `/v3/system/users/23f85921e67542a3a37b711d3ace53f8/roles/efc3726869ca4763809656e2e7e1646a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:22.555359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2df2e-2a43-43b5-90a1-ee8ec8b93f9d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=23f85921e67542a3a37b711d3ace53f8, role_id=efc3726869ca4763809656e2e7e1646a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:22.556801 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2df2e-2a43-43b5-90a1-ee8ec8b93f9d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:22.568785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 169/334] 10.222.0.26 () {58 vars in 1334 bytes} [Sat Sep 10 14:00:22 2022] PUT /identity/v3/system/users/23f85921e67542a3a37b711d3ace53f8/roles/efc3726869ca4763809656e2e7e1646a => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:24.447931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5593f8-f829-4c33-9a1f-3a7c9b7aba6f None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:24.448460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5593f8-f829-4c33-9a1f-3a7c9b7aba6f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:24.448460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5593f8-f829-4c33-9a1f-3a7c9b7aba6f None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:24.450265 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 166/335] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:24.459860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5812-1e2d-4058-b179-116606693704 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:24.460332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5812-1e2d-4058-b179-116606693704 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:24.460428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b5812-1e2d-4058-b179-116606693704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:24.494363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b37b5812-1e2d-4058-b179-116606693704 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:24.496367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b37b5812-1e2d-4058-b179-116606693704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:24.529757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:24.580534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed40f6-f83a-4c3f-9a32-828175252476 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:24.580883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed40f6-f83a-4c3f-9a32-828175252476 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:24.581139 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed40f6-f83a-4c3f-9a32-828175252476 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:24.582734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 167/336] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:24.591286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 170/337] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:24.596616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bde1c-4734-4bec-b5d3-fff339f43111 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:24.596899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bde1c-4734-4bec-b5d3-fff339f43111 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:24.597171 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bde1c-4734-4bec-b5d3-fff339f43111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:24.631859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-557bde1c-4734-4bec-b5d3-fff339f43111 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:24.633073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-557bde1c-4734-4bec-b5d3-fff339f43111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:24.714827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:24.737331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 168/338] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:25.007094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b5812-1e2d-4058-b179-116606693704 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:25.149625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b5812-1e2d-4058-b179-116606693704 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:25.161272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bde1c-4734-4bec-b5d3-fff339f43111 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:25.206469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fa177-2604-49fc-bc0c-a1d82bd86269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:25.207109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fa177-2604-49fc-bc0c-a1d82bd86269 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:25.207556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fa177-2604-49fc-bc0c-a1d82bd86269 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:25.207827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973fa177-2604-49fc-bc0c-a1d82bd86269 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:25.225509 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973fa177-2604-49fc-bc0c-a1d82bd86269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-973fa177-2604-49fc-bc0c-a1d82bd86269 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:25.230845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:25.232925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:25.234847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 171/339] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:25.241303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fa177-2604-49fc-bc0c-a1d82bd86269 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:25.277686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:25.389391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bde1c-4734-4bec-b5d3-fff339f43111 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:25.407715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973fa177-2604-49fc-bc0c-a1d82bd86269 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:25.461448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3302eb95-1f79-4072-b568-ec0a9d8eda82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:25.462088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302eb95-1f79-4072-b568-ec0a9d8eda82 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:25.462329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302eb95-1f79-4072-b568-ec0a9d8eda82 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:25.462598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3302eb95-1f79-4072-b568-ec0a9d8eda82 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:25.463292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302eb95-1f79-4072-b568-ec0a9d8eda82 None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:25.463609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302eb95-1f79-4072-b568-ec0a9d8eda82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:25.466145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3302eb95-1f79-4072-b568-ec0a9d8eda82 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:25.482183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f838dc0-aea6-4c23-98e1-430f2a7ca35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:25.482733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 172/340] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:25.483693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f838dc0-aea6-4c23-98e1-430f2a7ca35b None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:25.483800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74751ed3-e1d8-43a7-af96-cecc79f00e17 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:25.484044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74751ed3-e1d8-43a7-af96-cecc79f00e17 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:25.484312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74751ed3-e1d8-43a7-af96-cecc79f00e17 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:25.484663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f838dc0-aea6-4c23-98e1-430f2a7ca35b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:25.485143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f838dc0-aea6-4c23-98e1-430f2a7ca35b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:25.485564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 173/341] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:25.492603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74751ed3-e1d8-43a7-af96-cecc79f00e17 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:25.509120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f838dc0-aea6-4c23-98e1-430f2a7ca35b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-1f838dc0-aea6-4c23-98e1-430f2a7ca35b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:25.514035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:25.514930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:25.515777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 169/342] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:00:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:00:25.516462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c171f0-f39d-4521-828a-45badbd43407 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:25.517015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c171f0-f39d-4521-828a-45badbd43407 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:25.523492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c171f0-f39d-4521-828a-45badbd43407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:25.563416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-79c171f0-f39d-4521-828a-45badbd43407 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:25.564563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-79c171f0-f39d-4521-828a-45badbd43407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:25.634474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:25.647545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:25.676297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74751ed3-e1d8-43a7-af96-cecc79f00e17 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:25.687633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 170/343] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:25.688761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c171f0-f39d-4521-828a-45badbd43407 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:25.722431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b803e-4630-4f81-acd7-f977f7edcbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:25.723040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b803e-4630-4f81-acd7-f977f7edcbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:25.723313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b803e-4630-4f81-acd7-f977f7edcbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:25.723648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3b803e-4630-4f81-acd7-f977f7edcbb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:25.829914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c171f0-f39d-4521-828a-45badbd43407 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:25.859267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b803e-4630-4f81-acd7-f977f7edcbb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:25.861036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3b803e-4630-4f81-acd7-f977f7edcbb0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:25.872164 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fb919-35f3-4883-a1fa-eb6cf4646b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:25.873046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fb919-35f3-4883-a1fa-eb6cf4646b6b None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:25.873485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fb919-35f3-4883-a1fa-eb6cf4646b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:25.873891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fb919-35f3-4883-a1fa-eb6cf4646b6b None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:25.874705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fb919-35f3-4883-a1fa-eb6cf4646b6b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:25.875167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fb919-35f3-4883-a1fa-eb6cf4646b6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:25.876745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fb919-35f3-4883-a1fa-eb6cf4646b6b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:25.898256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 171/344] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:25.899788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247362e-a5e3-4c69-98fa-1a171d9929b8 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:25.900208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247362e-a5e3-4c69-98fa-1a171d9929b8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:25.900716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247362e-a5e3-4c69-98fa-1a171d9929b8 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:25.902059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 172/345] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:25.907628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e401e4f-a191-45ca-b6d9-d8a870f6fdc2 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:25.908112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e401e4f-a191-45ca-b6d9-d8a870f6fdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:25.908541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e401e4f-a191-45ca-b6d9-d8a870f6fdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:25.951513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-0e401e4f-a191-45ca-b6d9-d8a870f6fdc2 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:25.952938 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0e401e4f-a191-45ca-b6d9-d8a870f6fdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:25.955491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:25.979787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 174/346] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:25.980807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b803e-4630-4f81-acd7-f977f7edcbb0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:26.042941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:26.088886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3b803e-4630-4f81-acd7-f977f7edcbb0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:26.130136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18bb78-51b9-4267-bb88-4d7a625d29a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:26.131473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18bb78-51b9-4267-bb88-4d7a625d29a4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:26.131815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18bb78-51b9-4267-bb88-4d7a625d29a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:26.132084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb18bb78-51b9-4267-bb88-4d7a625d29a4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:26.148580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 173/347] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:26.150889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e401e4f-a191-45ca-b6d9-d8a870f6fdc2 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:26.188285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:26.215270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18bb78-51b9-4267-bb88-4d7a625d29a4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:26.217130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb18bb78-51b9-4267-bb88-4d7a625d29a4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:26.292365 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e401e4f-a191-45ca-b6d9-d8a870f6fdc2 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:26.360474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf2b11-86b7-475f-9785-532585cae39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:26.361037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf2b11-86b7-475f-9785-532585cae39c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:26.361309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf2b11-86b7-475f-9785-532585cae39c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:26.361577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bf2b11-86b7-475f-9785-532585cae39c None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:26.365626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf2b11-86b7-475f-9785-532585cae39c None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:26.367082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bf2b11-86b7-475f-9785-532585cae39c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:26.368770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 174/348] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:26 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:26.369667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf2b11-86b7-475f-9785-532585cae39c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:26.407812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 175/349] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:26.409013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18bb78-51b9-4267-bb88-4d7a625d29a4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:26.425123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:26.538271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bf2b11-86b7-475f-9785-532585cae39c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:26.578371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ca7b5-879f-47da-a814-0a9caf4d3a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:26.579019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ca7b5-879f-47da-a814-0a9caf4d3a14 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:26.579298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ca7b5-879f-47da-a814-0a9caf4d3a14 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:26.579607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ca7b5-879f-47da-a814-0a9caf4d3a14 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:26.593970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ca7b5-879f-47da-a814-0a9caf4d3a14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-761ca7b5-879f-47da-a814-0a9caf4d3a14 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:26.600961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:26.601874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:26.602697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 175/350] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:26.603372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ca7b5-879f-47da-a814-0a9caf4d3a14 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:26.606199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:26.640869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb18bb78-51b9-4267-bb88-4d7a625d29a4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:26.680231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333dd35-65eb-4019-8b62-57961d565caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:26.680870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333dd35-65eb-4019-8b62-57961d565caa None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:26.681143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333dd35-65eb-4019-8b62-57961d565caa None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:26.681411 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4333dd35-65eb-4019-8b62-57961d565caa None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:26.689521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333dd35-65eb-4019-8b62-57961d565caa None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:26.689521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4333dd35-65eb-4019-8b62-57961d565caa None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:26.691200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 176/351] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:26 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:26.696151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333dd35-65eb-4019-8b62-57961d565caa None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:26.714427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:26.725127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ca7b5-879f-47da-a814-0a9caf4d3a14 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:26.772159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcd4ba-dc1d-4385-a5ba-132c3d3bfdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:26.773123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcd4ba-dc1d-4385-a5ba-132c3d3bfdbd None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:26.773393 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcd4ba-dc1d-4385-a5ba-132c3d3bfdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:26.773658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fcd4ba-dc1d-4385-a5ba-132c3d3bfdbd None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:26.774861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcd4ba-dc1d-4385-a5ba-132c3d3bfdbd None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:26.777557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fcd4ba-dc1d-4385-a5ba-132c3d3bfdbd None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:26.779365 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 176/352] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:26 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:26.779991 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcd4ba-dc1d-4385-a5ba-132c3d3bfdbd None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:26.826275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4333dd35-65eb-4019-8b62-57961d565caa None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:26.878410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d338bb2-93ca-4e9c-8b7d-6b4dbfce7f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:26.879483 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d338bb2-93ca-4e9c-8b7d-6b4dbfce7f69 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:26.880039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d338bb2-93ca-4e9c-8b7d-6b4dbfce7f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:26.881262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d338bb2-93ca-4e9c-8b7d-6b4dbfce7f69 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:26.881976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d338bb2-93ca-4e9c-8b7d-6b4dbfce7f69 None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:26.882623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d338bb2-93ca-4e9c-8b7d-6b4dbfce7f69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:26.884815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d338bb2-93ca-4e9c-8b7d-6b4dbfce7f69 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:26.897512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 177/353] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:26.898655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad1b4-0e93-4058-9d01-ab84c37f2a63 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:26.898945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad1b4-0e93-4058-9d01-ab84c37f2a63 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:26.899219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55ad1b4-0e93-4058-9d01-ab84c37f2a63 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:26.900765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 178/354] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:26.901504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad1b4-0e93-4058-9d01-ab84c37f2a63 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:26.915953 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:26.917048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fcd4ba-dc1d-4385-a5ba-132c3d3bfdbd None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:26.955913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9d186-298d-434d-be97-46878213218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:26.956666 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9d186-298d-434d-be97-46878213218a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:26.956943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9d186-298d-434d-be97-46878213218a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:26.957224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef9d186-298d-434d-be97-46878213218a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:26.971814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef9d186-298d-434d-be97-46878213218a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-9ef9d186-298d-434d-be97-46878213218a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:26.979351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:26.980271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:26.981035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 177/355] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:26.981681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9d186-298d-434d-be97-46878213218a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:27.010047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:27.034252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55ad1b4-0e93-4058-9d01-ab84c37f2a63 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:27.071696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0ec57-01a7-441b-b03f-bf77853ff5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:27.072302 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0ec57-01a7-441b-b03f-bf77853ff5db None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:27.072572 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0ec57-01a7-441b-b03f-bf77853ff5db None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:27.072837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0ec57-01a7-441b-b03f-bf77853ff5db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:27.073573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0ec57-01a7-441b-b03f-bf77853ff5db None admin] RBAC: Adding query filter params (group.id=fd088c52d4e249a69f4917e21f9ce0c5, role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:27.073868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0ec57-01a7-441b-b03f-bf77853ff5db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:27.075310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c0ec57-01a7-441b-b03f-bf77853ff5db None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:27.081533 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 179/356] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 10 14:00:26 2022] GET /identity/v3/role_assignments?group.id=fd088c52d4e249a69f4917e21f9ce0c5&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:27.082461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0ec57-01a7-441b-b03f-bf77853ff5db None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:27.105292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef9d186-298d-434d-be97-46878213218a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:27.152337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b51399d-1dfe-4a49-8d99-5513350e5ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:27.153189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51399d-1dfe-4a49-8d99-5513350e5ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:27.153468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51399d-1dfe-4a49-8d99-5513350e5ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:27.155418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b51399d-1dfe-4a49-8d99-5513350e5ef8 None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:27.155418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51399d-1dfe-4a49-8d99-5513350e5ef8 None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:27.157148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b51399d-1dfe-4a49-8d99-5513350e5ef8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:27.158816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 178/357] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:26 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:27.160676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e7bbf-8464-423c-9317-4d14467560b9 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:27.161246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e7bbf-8464-423c-9317-4d14467560b9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:27.161716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9e7bbf-8464-423c-9317-4d14467560b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:27.188350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7e9e7bbf-8464-423c-9317-4d14467560b9 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:27.189934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7e9e7bbf-8464-423c-9317-4d14467560b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:27.194355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:27.238972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0ec57-01a7-441b-b03f-bf77853ff5db None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:27.284366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 179/358] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:27.285231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e7bbf-8464-423c-9317-4d14467560b9 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:27.285933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0467d-25a2-4e62-9a9b-2d7d3c22da9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:27.287699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0467d-25a2-4e62-9a9b-2d7d3c22da9c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:27.288169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0467d-25a2-4e62-9a9b-2d7d3c22da9c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:27.288579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0467d-25a2-4e62-9a9b-2d7d3c22da9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:27.308690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:27.396313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0467d-25a2-4e62-9a9b-2d7d3c22da9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:27.399074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0467d-25a2-4e62-9a9b-2d7d3c22da9c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:27.401598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9e7bbf-8464-423c-9317-4d14467560b9 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:27.450654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05059d-e0d0-4305-b855-0bca714fa924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:27.451654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05059d-e0d0-4305-b855-0bca714fa924 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:27.452182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05059d-e0d0-4305-b855-0bca714fa924 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:27.452598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05059d-e0d0-4305-b855-0bca714fa924 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:27.453429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05059d-e0d0-4305-b855-0bca714fa924 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:27.453881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05059d-e0d0-4305-b855-0bca714fa924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:27.456153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05059d-e0d0-4305-b855-0bca714fa924 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:27.466706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 180/359] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:27.468284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c34e-2212-4796-aba9-fd5508f3dbf1 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:27.469560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c34e-2212-4796-aba9-fd5508f3dbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:27.470009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c34e-2212-4796-aba9-fd5508f3dbf1 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:27.471778 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 181/360] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:27.472628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406c34e-2212-4796-aba9-fd5508f3dbf1 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:27.512814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:27.551184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 180/361] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:27.555467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0467d-25a2-4e62-9a9b-2d7d3c22da9c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:27.581249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:27.604793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406c34e-2212-4796-aba9-fd5508f3dbf1 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:27.641023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3058eb8-7861-425d-888f-ea969fda51e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:27.641885 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3058eb8-7861-425d-888f-ea969fda51e4 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:27.642165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3058eb8-7861-425d-888f-ea969fda51e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:27.642437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3058eb8-7861-425d-888f-ea969fda51e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:27.643169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3058eb8-7861-425d-888f-ea969fda51e4 None admin] RBAC: Adding query filter params (group.id=fd088c52d4e249a69f4917e21f9ce0c5, role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:27.643503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3058eb8-7861-425d-888f-ea969fda51e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:27.644948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3058eb8-7861-425d-888f-ea969fda51e4 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:27.651709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 182/362] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 10 14:00:27 2022] GET /identity/v3/role_assignments?group.id=fd088c52d4e249a69f4917e21f9ce0c5&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:27.652694 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81955966-8a37-4c93-8ea7-b3cca9bd0d5a None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:27.652979 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81955966-8a37-4c93-8ea7-b3cca9bd0d5a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:27.653248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81955966-8a37-4c93-8ea7-b3cca9bd0d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:27.674192 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-81955966-8a37-4c93-8ea7-b3cca9bd0d5a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:27.675389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-81955966-8a37-4c93-8ea7-b3cca9bd0d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:27.692657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0467d-25a2-4e62-9a9b-2d7d3c22da9c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:27.743272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f53ba-a4dc-47e4-bf0b-218964e04912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:27.744388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f53ba-a4dc-47e4-bf0b-218964e04912 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:27.744884 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f53ba-a4dc-47e4-bf0b-218964e04912 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:27.745358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f53ba-a4dc-47e4-bf0b-218964e04912 None admin] PATH_INFO: `/v3/users/23f85921e67542a3a37b711d3ace53f8` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:27.747729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f53ba-a4dc-47e4-bf0b-218964e04912 None admin] RBAC: Authorizing `identity:get_user(user_id=23f85921e67542a3a37b711d3ace53f8)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:27.749542 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f53ba-a4dc-47e4-bf0b-218964e04912 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:27.752809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 181/363] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:00:27 2022] GET /identity/v3/users/23f85921e67542a3a37b711d3ace53f8 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:27.754430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f53ba-a4dc-47e4-bf0b-218964e04912 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:27.767489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 183/364] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:27.767489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81955966-8a37-4c93-8ea7-b3cca9bd0d5a None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:27.813970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:27.915060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f53ba-a4dc-47e4-bf0b-218964e04912 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:27.916020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:27.952543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81955966-8a37-4c93-8ea7-b3cca9bd0d5a None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:27.971656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1b895-c523-45b6-8fb7-fa880e0077b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:27.972347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1b895-c523-45b6-8fb7-fa880e0077b5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:27.972626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1b895-c523-45b6-8fb7-fa880e0077b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:27.972898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b1b895-c523-45b6-8fb7-fa880e0077b5 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:27.974587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1b895-c523-45b6-8fb7-fa880e0077b5 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:27.976075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b1b895-c523-45b6-8fb7-fa880e0077b5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:27.977716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 182/365] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:27 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:27.978476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1b895-c523-45b6-8fb7-fa880e0077b5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:28.020246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeba2ed-cb09-4fcc-8c35-4fed7ef1d80a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:28.021266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeba2ed-cb09-4fcc-8c35-4fed7ef1d80a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:28.021751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeba2ed-cb09-4fcc-8c35-4fed7ef1d80a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:28.022207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeba2ed-cb09-4fcc-8c35-4fed7ef1d80a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:28.047504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeba2ed-cb09-4fcc-8c35-4fed7ef1d80a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-daeba2ed-cb09-4fcc-8c35-4fed7ef1d80a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:28.058138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:28.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:28.059908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:28.064177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 184/366] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:00:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:28.065003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeba2ed-cb09-4fcc-8c35-4fed7ef1d80a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:28.188008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b1b895-c523-45b6-8fb7-fa880e0077b5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:28.193565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeba2ed-cb09-4fcc-8c35-4fed7ef1d80a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:28.197991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:28.237992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fda61-4244-4360-9517-269f1f96fc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:28.238673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fda61-4244-4360-9517-269f1f96fc2a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:28.238942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fda61-4244-4360-9517-269f1f96fc2a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:28.239316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498fda61-4244-4360-9517-269f1f96fc2a None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:28.240858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fda61-4244-4360-9517-269f1f96fc2a None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:28.242304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498fda61-4244-4360-9517-269f1f96fc2a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:28.244596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 185/367] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:28 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:28.245458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fda61-4244-4360-9517-269f1f96fc2a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:28.248710 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b105047-38fe-41dd-9d25-afe64fc44fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:28.249329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b105047-38fe-41dd-9d25-afe64fc44fd8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:28.249603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b105047-38fe-41dd-9d25-afe64fc44fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:28.249875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b105047-38fe-41dd-9d25-afe64fc44fd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:28.250317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:28.250631 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b105047-38fe-41dd-9d25-afe64fc44fd8 None admin] RBAC: Adding query filter params (role.id=efc3726869ca4763809656e2e7e1646a, user.id=23f85921e67542a3a37b711d3ace53f8, scope.system=all) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:28.250925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b105047-38fe-41dd-9d25-afe64fc44fd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:28.253641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b105047-38fe-41dd-9d25-afe64fc44fd8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:28.274222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 183/368] 10.222.0.26 () {56 vars in 1308 bytes} [Sat Sep 10 14:00:27 2022] GET /identity/v3/role_assignments?user.id=23f85921e67542a3a37b711d3ace53f8&scope.system=all&role.id=efc3726869ca4763809656e2e7e1646a => generated 510 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:28.275069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b105047-38fe-41dd-9d25-afe64fc44fd8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:28.433279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498fda61-4244-4360-9517-269f1f96fc2a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:28.444226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b105047-38fe-41dd-9d25-afe64fc44fd8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:28.492637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d8da2-78d9-4965-991d-c932b7abfebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:28.493507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8da2-78d9-4965-991d-c932b7abfebf None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:28.493782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8da2-78d9-4965-991d-c932b7abfebf None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:28.494950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56d8da2-78d9-4965-991d-c932b7abfebf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:28.505462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edb1db-c100-43ba-875b-97c1aaa4e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:28.506367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edb1db-c100-43ba-875b-97c1aaa4e160 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:28.506685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edb1db-c100-43ba-875b-97c1aaa4e160 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:28.506954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7edb1db-c100-43ba-875b-97c1aaa4e160 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:28.507700 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edb1db-c100-43ba-875b-97c1aaa4e160 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:28.508176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edb1db-c100-43ba-875b-97c1aaa4e160 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:28.509545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7edb1db-c100-43ba-875b-97c1aaa4e160 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:28.511384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56d8da2-78d9-4965-991d-c932b7abfebf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:28.518017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 186/369] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a56d8da2-78d9-4965-991d-c932b7abfebf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:28.518755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:28.519637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:28.521017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:28.523537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 184/370] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:00:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:28.524292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edb1db-c100-43ba-875b-97c1aaa4e160 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:28.535805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d8da2-78d9-4965-991d-c932b7abfebf None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:28.569187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:28.603892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:28.657413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7edb1db-c100-43ba-875b-97c1aaa4e160 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:28.679797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56d8da2-78d9-4965-991d-c932b7abfebf None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:28.702835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d44fa-89a3-410b-8872-3f9c91a15cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:28.703838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d44fa-89a3-410b-8872-3f9c91a15cbc None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:28.704477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d44fa-89a3-410b-8872-3f9c91a15cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:28.704957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4d44fa-89a3-410b-8872-3f9c91a15cbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:28.705899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d44fa-89a3-410b-8872-3f9c91a15cbc None admin] RBAC: Adding query filter params (group.id=fd088c52d4e249a69f4917e21f9ce0c5, role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, scope.project.id=87ea676d839141b4b3e827be73b4e648) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:28.706413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d44fa-89a3-410b-8872-3f9c91a15cbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:28.708711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4d44fa-89a3-410b-8872-3f9c91a15cbc None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:28.716545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 187/371] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 10 14:00:28 2022] GET /identity/v3/role_assignments?group.id=fd088c52d4e249a69f4917e21f9ce0c5&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:28.718335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d44fa-89a3-410b-8872-3f9c91a15cbc None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:28.735435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abee2e6-a836-47f7-9584-ae7a56cb4048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:28.736137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abee2e6-a836-47f7-9584-ae7a56cb4048 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:28.736381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abee2e6-a836-47f7-9584-ae7a56cb4048 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:28.736653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abee2e6-a836-47f7-9584-ae7a56cb4048 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:28.737328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abee2e6-a836-47f7-9584-ae7a56cb4048 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:28.737626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abee2e6-a836-47f7-9584-ae7a56cb4048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:28.739016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abee2e6-a836-47f7-9584-ae7a56cb4048 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:28.768771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 185/372] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:28.769702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abee2e6-a836-47f7-9584-ae7a56cb4048 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:28.915148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4d44fa-89a3-410b-8872-3f9c91a15cbc None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:28.952217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:28.959510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed782e8-01ea-4119-ad18-6de885220a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:28.960135 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed782e8-01ea-4119-ad18-6de885220a78 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:28.960406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed782e8-01ea-4119-ad18-6de885220a78 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:28.960677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed782e8-01ea-4119-ad18-6de885220a78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:28.977326 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:29.002568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abee2e6-a836-47f7-9584-ae7a56cb4048 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:29.058209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed782e8-01ea-4119-ad18-6de885220a78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:29.059616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed782e8-01ea-4119-ad18-6de885220a78 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:29.092226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715916-be4d-4efe-81cb-117d7c849f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:29.092844 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715916-be4d-4efe-81cb-117d7c849f75 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:29.093124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715916-be4d-4efe-81cb-117d7c849f75 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:29.093393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f715916-be4d-4efe-81cb-117d7c849f75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:29.191218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f715916-be4d-4efe-81cb-117d7c849f75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:29.194391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f715916-be4d-4efe-81cb-117d7c849f75 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:29.244599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 188/373] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:29.250096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed782e8-01ea-4119-ad18-6de885220a78 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:29.296005 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:29.350538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:29.384772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 186/374] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:29.388681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715916-be4d-4efe-81cb-117d7c849f75 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:29.452644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed782e8-01ea-4119-ad18-6de885220a78 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:29.528926 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7949dc-b3fc-4279-b255-4447dab95c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:29.529561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7949dc-b3fc-4279-b255-4447dab95c52 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:29.529895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7949dc-b3fc-4279-b255-4447dab95c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:29.530168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7949dc-b3fc-4279-b255-4447dab95c52 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:29.531277 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7949dc-b3fc-4279-b255-4447dab95c52 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:29.534489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7949dc-b3fc-4279-b255-4447dab95c52 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:29.537186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 189/375] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:29 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:29.542969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7949dc-b3fc-4279-b255-4447dab95c52 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:29.564749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f715916-be4d-4efe-81cb-117d7c849f75 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:29.630361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77299e10-f642-456e-9644-1fbc22070eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:29.630915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77299e10-f642-456e-9644-1fbc22070eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:29.631186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77299e10-f642-456e-9644-1fbc22070eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:29.631484 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77299e10-f642-456e-9644-1fbc22070eb5 None admin] PATH_INFO: `/v3/projects/7a00398c85824c23b37ffe5fee90b840` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:29.633912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77299e10-f642-456e-9644-1fbc22070eb5 None admin] RBAC: Authorizing `identity:get_project(project_id=7a00398c85824c23b37ffe5fee90b840)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:29.635367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77299e10-f642-456e-9644-1fbc22070eb5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:29.642428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 187/376] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:29 2022] GET /identity/v3/projects/7a00398c85824c23b37ffe5fee90b840 => generated 344 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:29.645792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77299e10-f642-456e-9644-1fbc22070eb5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:29.731919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:29.738612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:29.744388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7949dc-b3fc-4279-b255-4447dab95c52 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:29.798527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884068ca-7b6d-441f-b66f-0695dffca7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:29.799403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884068ca-7b6d-441f-b66f-0695dffca7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:29.799864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884068ca-7b6d-441f-b66f-0695dffca7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:29.800275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884068ca-7b6d-441f-b66f-0695dffca7e2 None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:29.800612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77299e10-f642-456e-9644-1fbc22070eb5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:29.801871 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884068ca-7b6d-441f-b66f-0695dffca7e2 None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:29.804685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884068ca-7b6d-441f-b66f-0695dffca7e2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:29.806552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 190/377] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:29 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:29.811883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dabf5-1e70-404a-b0c3-cb5e7baf41bd None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:29.812773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dabf5-1e70-404a-b0c3-cb5e7baf41bd None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:29.813189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47dabf5-1e70-404a-b0c3-cb5e7baf41bd None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:29.815610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 191/378] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:29.819398 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dabf5-1e70-404a-b0c3-cb5e7baf41bd None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:29.854286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1a9b97-85e6-4326-ae1c-8c7efd8c7c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:29.855819 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a9b97-85e6-4326-ae1c-8c7efd8c7c64 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:29.856261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a9b97-85e6-4326-ae1c-8c7efd8c7c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:29.856912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1a9b97-85e6-4326-ae1c-8c7efd8c7c64 None admin] PATH_INFO: `/v3/groups/50a50b766d144d65a355c672d440a33e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:29.865301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a9b97-85e6-4326-ae1c-8c7efd8c7c64 None admin] RBAC: Authorizing `identity:get_group(group_id=50a50b766d144d65a355c672d440a33e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:29.867018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1a9b97-85e6-4326-ae1c-8c7efd8c7c64 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:29.875359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 188/379] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:29 2022] GET /identity/v3/groups/50a50b766d144d65a355c672d440a33e => generated 229 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:29.876427 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b3be2-b8ba-480b-b2cc-8f5929c9d114 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:29.876622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b3be2-b8ba-480b-b2cc-8f5929c9d114 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:29.876857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b3be2-b8ba-480b-b2cc-8f5929c9d114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:29.903248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7b7b3be2-b8ba-480b-b2cc-8f5929c9d114 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:29.905513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7b7b3be2-b8ba-480b-b2cc-8f5929c9d114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:29.955659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47dabf5-1e70-404a-b0c3-cb5e7baf41bd None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:30.005005 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5f22e-c77d-4820-94ee-038cae44b012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:30.006158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5f22e-c77d-4820-94ee-038cae44b012 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:30.006438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 189/380] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:30.007332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b3be2-b8ba-480b-b2cc-8f5929c9d114 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:30.007838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5f22e-c77d-4820-94ee-038cae44b012 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:30.008304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5f22e-c77d-4820-94ee-038cae44b012 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:30.009388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5f22e-c77d-4820-94ee-038cae44b012 None admin] RBAC: Adding query filter params (group.id=fd088c52d4e249a69f4917e21f9ce0c5, role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf, scope.project.id=87ea676d839141b4b3e827be73b4e648) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:30.009894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5f22e-c77d-4820-94ee-038cae44b012 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:30.011573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5f22e-c77d-4820-94ee-038cae44b012 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:30.026138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 192/381] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 10 14:00:29 2022] GET /identity/v3/role_assignments?group.id=fd088c52d4e249a69f4917e21f9ce0c5&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 260 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:30.081674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:30.150059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b3be2-b8ba-480b-b2cc-8f5929c9d114 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:30.194262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefc320b-7803-471c-8c12-ca78a55d4ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:30.195232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc320b-7803-471c-8c12-ca78a55d4ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:30.195725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc320b-7803-471c-8c12-ca78a55d4ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:30.196190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefc320b-7803-471c-8c12-ca78a55d4ef4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:30.197154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc320b-7803-471c-8c12-ca78a55d4ef4 None admin] RBAC: Adding query filter params (group.id=50a50b766d144d65a355c672d440a33e, role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=7a00398c85824c23b37ffe5fee90b840) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:30.197709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc320b-7803-471c-8c12-ca78a55d4ef4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:30.199426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefc320b-7803-471c-8c12-ca78a55d4ef4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:30.211642 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 190/382] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 10 14:00:30 2022] GET /identity/v3/role_assignments?group.id=50a50b766d144d65a355c672d440a33e&scope.project.id=7a00398c85824c23b37ffe5fee90b840&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:30.332455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5f22e-c77d-4820-94ee-038cae44b012 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:30.370843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:30.468726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5f22e-c77d-4820-94ee-038cae44b012 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:30.509966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9359e-e69e-4b56-b0b4-1681344c2e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:30.510613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9359e-e69e-4b56-b0b4-1681344c2e6c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:30.510885 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9359e-e69e-4b56-b0b4-1681344c2e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:30.511151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9359e-e69e-4b56-b0b4-1681344c2e6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:30.527532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9359e-e69e-4b56-b0b4-1681344c2e6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-8fa9359e-e69e-4b56-b0b4-1681344c2e6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:30.532879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:30.533731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:30.534494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 193/383] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:30.541150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7b395-b3a2-4654-93a5-7d1e0524b5e4 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:30.541432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7b395-b3a2-4654-93a5-7d1e0524b5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:30.541704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7b395-b3a2-4654-93a5-7d1e0524b5e4 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:30.543110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 191/384] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:30.544115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9359e-e69e-4b56-b0b4-1681344c2e6c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:30.551523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04013963-78dc-4a3d-9211-20bcbf17d41b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:30.551523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04013963-78dc-4a3d-9211-20bcbf17d41b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:30.551523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04013963-78dc-4a3d-9211-20bcbf17d41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:30.588479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-04013963-78dc-4a3d-9211-20bcbf17d41b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:30.590139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-04013963-78dc-4a3d-9211-20bcbf17d41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:30.648716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:30.662927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9359e-e69e-4b56-b0b4-1681344c2e6c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:30.701023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e078c-1bc5-414c-838c-2ce8b820f863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:30.701630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e078c-1bc5-414c-838c-2ce8b820f863 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:30.701902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e078c-1bc5-414c-838c-2ce8b820f863 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:30.702168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e078c-1bc5-414c-838c-2ce8b820f863 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:30.702823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e078c-1bc5-414c-838c-2ce8b820f863 None admin] RBAC: Adding query filter params (name=member) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:30.703119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e078c-1bc5-414c-838c-2ce8b820f863 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:30.704531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57e078c-1bc5-414c-838c-2ce8b820f863 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:30.714989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 194/385] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:30.721357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e078c-1bc5-414c-838c-2ce8b820f863 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:30.769065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 192/386] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:30.829545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e078c-1bc5-414c-838c-2ce8b820f863 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:30.866283 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7faa2e4-fae5-4d0b-bdf6-735e23e5b29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:30.866917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7faa2e4-fae5-4d0b-bdf6-735e23e5b29a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:30.867184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7faa2e4-fae5-4d0b-bdf6-735e23e5b29a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:30.867474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7faa2e4-fae5-4d0b-bdf6-735e23e5b29a None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:30.868628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7faa2e4-fae5-4d0b-bdf6-735e23e5b29a None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:30.870061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7faa2e4-fae5-4d0b-bdf6-735e23e5b29a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:30.875017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 195/387] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:30 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:30.877718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04013963-78dc-4a3d-9211-20bcbf17d41b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:30.929598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:30.994805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04013963-78dc-4a3d-9211-20bcbf17d41b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:31.033389 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1dff19-8c6f-4718-ba01-fee6b4dc17cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:31.034383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dff19-8c6f-4718-ba01-fee6b4dc17cd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:31.034856 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dff19-8c6f-4718-ba01-fee6b4dc17cd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:31.035295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1dff19-8c6f-4718-ba01-fee6b4dc17cd None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:31.037408 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1dff19-8c6f-4718-ba01-fee6b4dc17cd None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:31.039042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1dff19-8c6f-4718-ba01-fee6b4dc17cd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:31.040929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 193/388] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:30 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:31.048485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7faa2e4-fae5-4d0b-bdf6-735e23e5b29a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:31.073134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:31.107634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1dff19-8c6f-4718-ba01-fee6b4dc17cd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:31.219587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7faa2e4-fae5-4d0b-bdf6-735e23e5b29a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:31.289329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e149646-f8ba-4564-8002-bf174038b3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:31.291227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e149646-f8ba-4564-8002-bf174038b3c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:31.291520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e149646-f8ba-4564-8002-bf174038b3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:31.291793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e149646-f8ba-4564-8002-bf174038b3c9 None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/groups/fd088c52d4e249a69f4917e21f9ce0c5/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:31.296897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e149646-f8ba-4564-8002-bf174038b3c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c95a63dd5554fe1b35750e8ec50d808, group_id=fd088c52d4e249a69f4917e21f9ce0c5, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:31.299170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e149646-f8ba-4564-8002-bf174038b3c9 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:31.309626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1dff19-8c6f-4718-ba01-fee6b4dc17cd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:31.328471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 196/389] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Sep 10 14:00:31 2022] PUT /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/groups/fd088c52d4e249a69f4917e21f9ce0c5/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:31.337013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:31.368169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2fe28-7124-481f-a86c-b8000471e1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:31.368924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2fe28-7124-481f-a86c-b8000471e1fe None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:31.369202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2fe28-7124-481f-a86c-b8000471e1fe None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:31.369471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa2fe28-7124-481f-a86c-b8000471e1fe None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:31.383119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa2fe28-7124-481f-a86c-b8000471e1fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-3aa2fe28-7124-481f-a86c-b8000471e1fe None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:31.388555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:31.389463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:31.390271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:31.393231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 194/390] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:00:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:00:31.396598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e149646-f8ba-4564-8002-bf174038b3c9 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:31.511822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:31.539116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e149646-f8ba-4564-8002-bf174038b3c9 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:31.588936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786156ac-78d8-411b-abdf-b234f66215da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:31.589890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786156ac-78d8-411b-abdf-b234f66215da None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:31.590173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786156ac-78d8-411b-abdf-b234f66215da None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:31.590443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786156ac-78d8-411b-abdf-b234f66215da None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:31.591135 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786156ac-78d8-411b-abdf-b234f66215da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:31.591431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786156ac-78d8-411b-abdf-b234f66215da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:31.593039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786156ac-78d8-411b-abdf-b234f66215da None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:31.603315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 197/391] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:31.608814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2fe28-7124-481f-a86c-b8000471e1fe None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:31.706045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e444f9-f52d-4bf0-bc04-94ea9622449f None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:31.706379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e444f9-f52d-4bf0-bc04-94ea9622449f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:31.706585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e444f9-f52d-4bf0-bc04-94ea9622449f None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:31.710390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 198/392] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:00:31.715343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9e742-e2ff-4b35-b208-af516a07f1ba None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:31.715789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9e742-e2ff-4b35-b208-af516a07f1ba None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:31.716185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9e742-e2ff-4b35-b208-af516a07f1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:31.729125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa2fe28-7124-481f-a86c-b8000471e1fe None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:31.735928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-70a9e742-e2ff-4b35-b208-af516a07f1ba None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:31.737215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-70a9e742-e2ff-4b35-b208-af516a07f1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:31.785175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7ed54-9a4e-417c-bd0a-2c65feeae851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:31.785847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7ed54-9a4e-417c-bd0a-2c65feeae851 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:31.786114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7ed54-9a4e-417c-bd0a-2c65feeae851 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:31.786563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7ed54-9a4e-417c-bd0a-2c65feeae851 None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:31.792124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7ed54-9a4e-417c-bd0a-2c65feeae851 None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:31.793580 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7ed54-9a4e-417c-bd0a-2c65feeae851 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:31.795506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 195/393] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:31 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:31.800496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7ed54-9a4e-417c-bd0a-2c65feeae851 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:31.810345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 199/394] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:31.859900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:31.923780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7ed54-9a4e-417c-bd0a-2c65feeae851 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:31.959762 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43875157-8ef4-4d03-8190-b3215cefadf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:31.960486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43875157-8ef4-4d03-8190-b3215cefadf6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:31.960761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43875157-8ef4-4d03-8190-b3215cefadf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:31.961091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43875157-8ef4-4d03-8190-b3215cefadf6 None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:31.962276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43875157-8ef4-4d03-8190-b3215cefadf6 None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:31.963735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43875157-8ef4-4d03-8190-b3215cefadf6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:31.965698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 196/395] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:31 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:31.972269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9e742-e2ff-4b35-b208-af516a07f1ba None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:32.074004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:32.115765 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a9e742-e2ff-4b35-b208-af516a07f1ba None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:32.130553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43875157-8ef4-4d03-8190-b3215cefadf6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:32.155266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc29b2b-2175-432d-91c0-3f8d335c40fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:32.155960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc29b2b-2175-432d-91c0-3f8d335c40fa None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:32.156242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc29b2b-2175-432d-91c0-3f8d335c40fa None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:32.156516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc29b2b-2175-432d-91c0-3f8d335c40fa None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/groups/fd088c52d4e249a69f4917e21f9ce0c5/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:32.158659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc29b2b-2175-432d-91c0-3f8d335c40fa None admin] RBAC: Authorizing `identity:create_grant(project_id=2c95a63dd5554fe1b35750e8ec50d808, group_id=fd088c52d4e249a69f4917e21f9ce0c5, role_id=9c8bfff8d1a843dbb8aa6482c9fa35bf)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:32.160145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc29b2b-2175-432d-91c0-3f8d335c40fa None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:32.178121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 200/396] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Sep 10 14:00:31 2022] PUT /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808/groups/fd088c52d4e249a69f4917e21f9ce0c5/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:32.315977 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43875157-8ef4-4d03-8190-b3215cefadf6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:32.330597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:32.373339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c86006-1e2d-4fc6-9e59-5d7f17415692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:32.373912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c86006-1e2d-4fc6-9e59-5d7f17415692 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:32.374187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c86006-1e2d-4fc6-9e59-5d7f17415692 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:32.374456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c86006-1e2d-4fc6-9e59-5d7f17415692 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:32.390958 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c86006-1e2d-4fc6-9e59-5d7f17415692 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-31c86006-1e2d-4fc6-9e59-5d7f17415692 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:32.396402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:32.397266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:32.398052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 197/397] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:32.403746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc29b2b-2175-432d-91c0-3f8d335c40fa None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:32.530534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc29b2b-2175-432d-91c0-3f8d335c40fa None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:32.560698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:32.571521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c18d41-92b8-4964-9a44-76764bc8fc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:32.572207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c18d41-92b8-4964-9a44-76764bc8fc07 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:32.572510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c18d41-92b8-4964-9a44-76764bc8fc07 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:32.572852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c18d41-92b8-4964-9a44-76764bc8fc07 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:32.573531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c18d41-92b8-4964-9a44-76764bc8fc07 None admin] RBAC: Adding query filter params (name=member) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:32.573834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c18d41-92b8-4964-9a44-76764bc8fc07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:32.575205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c18d41-92b8-4964-9a44-76764bc8fc07 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:32.589410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 201/398] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:32.595320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c86006-1e2d-4fc6-9e59-5d7f17415692 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:32.737948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c86006-1e2d-4fc6-9e59-5d7f17415692 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:32.793749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97469-05f1-42b8-bbd9-5e69aa0b6c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:32.794390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97469-05f1-42b8-bbd9-5e69aa0b6c29 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:32.794719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97469-05f1-42b8-bbd9-5e69aa0b6c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:32.795003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f97469-05f1-42b8-bbd9-5e69aa0b6c29 None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:32.796196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f97469-05f1-42b8-bbd9-5e69aa0b6c29 None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:32.797662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f97469-05f1-42b8-bbd9-5e69aa0b6c29 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:32.799422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 198/399] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:32 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:32.805634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c18d41-92b8-4964-9a44-76764bc8fc07 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:32.924070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c18d41-92b8-4964-9a44-76764bc8fc07 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:32.962745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2f249-cb67-455a-ab55-d43ccdc72c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:32.963403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f249-cb67-455a-ab55-d43ccdc72c08 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:32.963934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f249-cb67-455a-ab55-d43ccdc72c08 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:32.964448 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2f249-cb67-455a-ab55-d43ccdc72c08 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:32.965845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2f249-cb67-455a-ab55-d43ccdc72c08 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:32.968445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2f249-cb67-455a-ab55-d43ccdc72c08 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:32.971976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 202/400] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:32 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:32.979931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97469-05f1-42b8-bbd9-5e69aa0b6c29 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:33.028355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:33.131760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f97469-05f1-42b8-bbd9-5e69aa0b6c29 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:33.169056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc429a0-0f8e-4c8c-8292-022e47e34273 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:33.170332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc429a0-0f8e-4c8c-8292-022e47e34273 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:33.171039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc429a0-0f8e-4c8c-8292-022e47e34273 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:33.173851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 203/401] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:33.180578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9bb63-42a3-4f0c-8b62-b019b937237a None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:33.180853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9bb63-42a3-4f0c-8b62-b019b937237a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:33.181122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b9bb63-42a3-4f0c-8b62-b019b937237a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:33.199092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cdaacb-376c-4ab3-8f94-321a64d730b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:33.199789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdaacb-376c-4ab3-8f94-321a64d730b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:33.200065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdaacb-376c-4ab3-8f94-321a64d730b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:33.200334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cdaacb-376c-4ab3-8f94-321a64d730b1 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648/groups/fd088c52d4e249a69f4917e21f9ce0c5/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:33.202336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdaacb-376c-4ab3-8f94-321a64d730b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=87ea676d839141b4b3e827be73b4e648, group_id=fd088c52d4e249a69f4917e21f9ce0c5, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:33.203840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cdaacb-376c-4ab3-8f94-321a64d730b1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:33.210862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c3b9bb63-42a3-4f0c-8b62-b019b937237a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:33.212203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c3b9bb63-42a3-4f0c-8b62-b019b937237a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:33.228098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 199/402] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Sep 10 14:00:32 2022] PUT /identity/v3/projects/87ea676d839141b4b3e827be73b4e648/groups/fd088c52d4e249a69f4917e21f9ce0c5/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:33.229932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643e1b0-c53b-4b92-b9dc-92977c577d89 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:33.230461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643e1b0-c53b-4b92-b9dc-92977c577d89 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:33.230973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9643e1b0-c53b-4b92-b9dc-92977c577d89 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:33.232713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 200/403] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:33.238324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc820ff9-d81e-4b8e-a612-b47d756f651f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:33.238624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc820ff9-d81e-4b8e-a612-b47d756f651f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:33.238874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc820ff9-d81e-4b8e-a612-b47d756f651f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:33.263531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fc820ff9-d81e-4b8e-a612-b47d756f651f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:33.264832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fc820ff9-d81e-4b8e-a612-b47d756f651f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:33.347951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:33.373546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 204/404] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:33.409278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 201/405] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:33.697359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9bb63-42a3-4f0c-8b62-b019b937237a None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:33.796870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc820ff9-d81e-4b8e-a612-b47d756f651f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:33.830610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b9bb63-42a3-4f0c-8b62-b019b937237a None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:33.833354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:33.900271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee71e4-cf08-4039-9c11-8b50721359e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:33.900914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee71e4-cf08-4039-9c11-8b50721359e1 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:33.901200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee71e4-cf08-4039-9c11-8b50721359e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:33.901472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbee71e4-cf08-4039-9c11-8b50721359e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:33.925569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbee71e4-cf08-4039-9c11-8b50721359e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-bbee71e4-cf08-4039-9c11-8b50721359e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:33.931548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:33.932409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:33.933309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 205/406] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:00:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:33.938969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee71e4-cf08-4039-9c11-8b50721359e1 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:33.985356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc820ff9-d81e-4b8e-a612-b47d756f651f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:33.997534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:34.039611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358c00c-b2a0-48df-b6ec-8f5a4e246b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:34.040254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358c00c-b2a0-48df-b6ec-8f5a4e246b40 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:34.040531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358c00c-b2a0-48df-b6ec-8f5a4e246b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:34.040801 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358c00c-b2a0-48df-b6ec-8f5a4e246b40 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:34.061999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358c00c-b2a0-48df-b6ec-8f5a4e246b40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-8358c00c-b2a0-48df-b6ec-8f5a4e246b40 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:34.066050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:34.066934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:34.068690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 202/407] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:00:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:34.081156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358c00c-b2a0-48df-b6ec-8f5a4e246b40 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:34.097894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbee71e4-cf08-4039-9c11-8b50721359e1 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:34.142276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554fbb6-41f5-461d-b0f5-93d60f3798de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:34.143998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554fbb6-41f5-461d-b0f5-93d60f3798de None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:34.148769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554fbb6-41f5-461d-b0f5-93d60f3798de None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:34.149303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1554fbb6-41f5-461d-b0f5-93d60f3798de None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:34.150180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554fbb6-41f5-461d-b0f5-93d60f3798de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:34.150681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554fbb6-41f5-461d-b0f5-93d60f3798de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:34.152302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1554fbb6-41f5-461d-b0f5-93d60f3798de None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:34.168833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 206/408] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:34.173180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554fbb6-41f5-461d-b0f5-93d60f3798de None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:34.253540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:34.277416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358c00c-b2a0-48df-b6ec-8f5a4e246b40 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:34.353094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8c373-bd0d-48cc-82ac-453700444d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:34.353707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8c373-bd0d-48cc-82ac-453700444d95 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:34.353988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8c373-bd0d-48cc-82ac-453700444d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:34.354261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c8c373-bd0d-48cc-82ac-453700444d95 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:34.354929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8c373-bd0d-48cc-82ac-453700444d95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:34.355229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8c373-bd0d-48cc-82ac-453700444d95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:34.356662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c8c373-bd0d-48cc-82ac-453700444d95 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:34.369756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1554fbb6-41f5-461d-b0f5-93d60f3798de None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:34.384305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 203/409] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:34.385445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8c373-bd0d-48cc-82ac-453700444d95 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:34.404890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:34.433543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f5ad7-6bd6-4e7d-99f2-4783dbae0232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:34.434178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f5ad7-6bd6-4e7d-99f2-4783dbae0232 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:34.434452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f5ad7-6bd6-4e7d-99f2-4783dbae0232 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:34.434722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f5ad7-6bd6-4e7d-99f2-4783dbae0232 None admin] PATH_INFO: `/v3/groups/50a50b766d144d65a355c672d440a33e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:34.435872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f5ad7-6bd6-4e7d-99f2-4783dbae0232 None admin] RBAC: Authorizing `identity:get_group(group_id=50a50b766d144d65a355c672d440a33e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:34.437336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f5ad7-6bd6-4e7d-99f2-4783dbae0232 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:34.438916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 207/410] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:34 2022] GET /identity/v3/groups/50a50b766d144d65a355c672d440a33e => generated 229 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:34.440066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc92197-86a3-4dee-ae81-2a50ebf59004 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:34.440671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc92197-86a3-4dee-ae81-2a50ebf59004 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:34.440671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc92197-86a3-4dee-ae81-2a50ebf59004 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:34.441764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 208/411] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:34 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:34.448379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc92197-86a3-4dee-ae81-2a50ebf59004 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:34.528080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c8c373-bd0d-48cc-82ac-453700444d95 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:34.562296 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc92197-86a3-4dee-ae81-2a50ebf59004 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:34.564136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd807b6-ea91-44ce-b4a4-e92721a27e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:34.564888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd807b6-ea91-44ce-b4a4-e92721a27e5f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:34.565168 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd807b6-ea91-44ce-b4a4-e92721a27e5f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:34.565454 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd807b6-ea91-44ce-b4a4-e92721a27e5f None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:34.566564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd807b6-ea91-44ce-b4a4-e92721a27e5f None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:34.568335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd807b6-ea91-44ce-b4a4-e92721a27e5f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:34.570096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 204/412] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:34 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:34.571367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1bead-4392-40e7-87aa-ab5a916a41a6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:34.571664 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1bead-4392-40e7-87aa-ab5a916a41a6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:34.571939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f1bead-4392-40e7-87aa-ab5a916a41a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:34.596437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-88f1bead-4392-40e7-87aa-ab5a916a41a6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:34.597739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-88f1bead-4392-40e7-87aa-ab5a916a41a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:34.608891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:34.610535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f2a6a-77d8-40eb-9051-c5b0eeb60bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:34.611250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f2a6a-77d8-40eb-9051-c5b0eeb60bbf None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:34.611899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f2a6a-77d8-40eb-9051-c5b0eeb60bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:34.612183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04f2a6a-77d8-40eb-9051-c5b0eeb60bbf None admin] PATH_INFO: `/v3/projects/7a00398c85824c23b37ffe5fee90b840` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:34.613343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f2a6a-77d8-40eb-9051-c5b0eeb60bbf None admin] RBAC: Authorizing `identity:get_project(project_id=7a00398c85824c23b37ffe5fee90b840)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:34.615139 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04f2a6a-77d8-40eb-9051-c5b0eeb60bbf None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:34.618024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 209/413] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:34 2022] GET /identity/v3/projects/7a00398c85824c23b37ffe5fee90b840 => generated 344 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:34.618883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f2a6a-77d8-40eb-9051-c5b0eeb60bbf None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:34.674722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 205/414] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:34.675919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1bead-4392-40e7-87aa-ab5a916a41a6 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:34.690415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:34.795390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04f2a6a-77d8-40eb-9051-c5b0eeb60bbf None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:34.863412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebc5a3-1700-4285-968d-72301db20222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:34.863941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f1bead-4392-40e7-87aa-ab5a916a41a6 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:34.866501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebc5a3-1700-4285-968d-72301db20222 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:34.867023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebc5a3-1700-4285-968d-72301db20222 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:34.867538 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beebc5a3-1700-4285-968d-72301db20222 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:34.869783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebc5a3-1700-4285-968d-72301db20222 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:34.871598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beebc5a3-1700-4285-968d-72301db20222 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:34.878133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 210/415] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:34 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:34.888135 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebc5a3-1700-4285-968d-72301db20222 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:34.969725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83426b-33b0-46b5-bcbf-c17d5e2408d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:34.970070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83426b-33b0-46b5-bcbf-c17d5e2408d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:34.970292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83426b-33b0-46b5-bcbf-c17d5e2408d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:34.970564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb83426b-33b0-46b5-bcbf-c17d5e2408d7 None admin] PATH_INFO: `/v3/projects/7a00398c85824c23b37ffe5fee90b840/groups/50a50b766d144d65a355c672d440a33e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:34.976707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83426b-33b0-46b5-bcbf-c17d5e2408d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=7a00398c85824c23b37ffe5fee90b840, group_id=50a50b766d144d65a355c672d440a33e, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:34.978186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb83426b-33b0-46b5-bcbf-c17d5e2408d7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:35.005227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beebc5a3-1700-4285-968d-72301db20222 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:35.008908 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 206/416] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Sep 10 14:00:34 2022] PUT /identity/v3/projects/7a00398c85824c23b37ffe5fee90b840/groups/50a50b766d144d65a355c672d440a33e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:35.011561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83426b-33b0-46b5-bcbf-c17d5e2408d7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:35.015973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:35.035060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:35.077804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de46854-923e-4394-abc5-91bb62c3575d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:35.079004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de46854-923e-4394-abc5-91bb62c3575d None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:35.079431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de46854-923e-4394-abc5-91bb62c3575d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:35.079893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de46854-923e-4394-abc5-91bb62c3575d None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648/groups/fd088c52d4e249a69f4917e21f9ce0c5/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:35.082492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de46854-923e-4394-abc5-91bb62c3575d None admin] RBAC: Authorizing `identity:create_grant(project_id=87ea676d839141b4b3e827be73b4e648, group_id=fd088c52d4e249a69f4917e21f9ce0c5, role_id=9c8bfff8d1a843dbb8aa6482c9fa35bf)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:35.085309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de46854-923e-4394-abc5-91bb62c3575d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:35.103352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 211/417] 10.222.0.26 () {58 vars in 1442 bytes} [Sat Sep 10 14:00:34 2022] PUT /identity/v3/projects/87ea676d839141b4b3e827be73b4e648/groups/fd088c52d4e249a69f4917e21f9ce0c5/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:35.112734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c07df-9a3e-48f2-b375-390a1b18cefd None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:35.113354 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c07df-9a3e-48f2-b375-390a1b18cefd None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:35.113814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c07df-9a3e-48f2-b375-390a1b18cefd None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:35.116190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 212/418] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:35 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:35.122042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b0683-8d0a-4060-8549-06866a7d5c26 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:35.122648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b0683-8d0a-4060-8549-06866a7d5c26 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:35.123151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b0683-8d0a-4060-8549-06866a7d5c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:35.153087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb83426b-33b0-46b5-bcbf-c17d5e2408d7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:35.154164 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7a7b0683-8d0a-4060-8549-06866a7d5c26 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:35.155611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7a7b0683-8d0a-4060-8549-06866a7d5c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:35.196444 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2623a0-77bf-45dc-95fa-6f6d882fdd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:35.197096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2623a0-77bf-45dc-95fa-6f6d882fdd1c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:35.197380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2623a0-77bf-45dc-95fa-6f6d882fdd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:35.197708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2623a0-77bf-45dc-95fa-6f6d882fdd1c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:35.211660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2623a0-77bf-45dc-95fa-6f6d882fdd1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-df2623a0-77bf-45dc-95fa-6f6d882fdd1c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:35.217841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:35.218789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:35.220305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 207/419] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:35.226593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2623a0-77bf-45dc-95fa-6f6d882fdd1c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:35.257414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 213/420] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:35.314624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:35.365644 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2623a0-77bf-45dc-95fa-6f6d882fdd1c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:35.405991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92c31d-e1c5-46a1-a3a7-bbe905b5717c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:35.406661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92c31d-e1c5-46a1-a3a7-bbe905b5717c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:35.406945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92c31d-e1c5-46a1-a3a7-bbe905b5717c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:35.407214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f92c31d-e1c5-46a1-a3a7-bbe905b5717c None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:35.407915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92c31d-e1c5-46a1-a3a7-bbe905b5717c None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:35.408221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92c31d-e1c5-46a1-a3a7-bbe905b5717c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:35.409602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f92c31d-e1c5-46a1-a3a7-bbe905b5717c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:35.418880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 208/421] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:35.424719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b0683-8d0a-4060-8549-06866a7d5c26 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:35.531789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b0683-8d0a-4060-8549-06866a7d5c26 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:35.541177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:35.558342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92c31d-e1c5-46a1-a3a7-bbe905b5717c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:35.580720 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdb3887-a985-4029-ba30-9ec168b8c0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:35.581332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb3887-a985-4029-ba30-9ec168b8c0fd None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:35.581614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb3887-a985-4029-ba30-9ec168b8c0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:35.581882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdb3887-a985-4029-ba30-9ec168b8c0fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:35.668599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb3887-a985-4029-ba30-9ec168b8c0fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:35.670097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdb3887-a985-4029-ba30-9ec168b8c0fd None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:35.735811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f92c31d-e1c5-46a1-a3a7-bbe905b5717c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:35.787284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:35.804590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf239dc7-1f26-4182-931f-48737ad9badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:35.805226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf239dc7-1f26-4182-931f-48737ad9badc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:35.805499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf239dc7-1f26-4182-931f-48737ad9badc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:35.805763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf239dc7-1f26-4182-931f-48737ad9badc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:35.829081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf239dc7-1f26-4182-931f-48737ad9badc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-bf239dc7-1f26-4182-931f-48737ad9badc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:35.838233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:35.839129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:35.840431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 209/422] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:00:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:35.843901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bdf0a-3a51-48b8-80bd-a2d0fb9cb5de None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:35.844187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bdf0a-3a51-48b8-80bd-a2d0fb9cb5de None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:35.844717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4bdf0a-3a51-48b8-80bd-a2d0fb9cb5de None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:35.846236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 210/423] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:35.846950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bdf0a-3a51-48b8-80bd-a2d0fb9cb5de None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:35.857642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 214/424] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:35.859214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1a98f-a599-45b5-b528-9005f098f931 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:35.859751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1a98f-a599-45b5-b528-9005f098f931 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:35.860219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df1a98f-a599-45b5-b528-9005f098f931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:35.874219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:35.881500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-3df1a98f-a599-45b5-b528-9005f098f931 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:35.882949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3df1a98f-a599-45b5-b528-9005f098f931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:35.957610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 215/425] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:35.959190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1a98f-a599-45b5-b528-9005f098f931 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:35.992225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4bdf0a-3a51-48b8-80bd-a2d0fb9cb5de None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:36.079192 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df1a98f-a599-45b5-b528-9005f098f931 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:36.085663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d13f4f-1496-478d-a0e2-f775385c3fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:36.086275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d13f4f-1496-478d-a0e2-f775385c3fcc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:36.086556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d13f4f-1496-478d-a0e2-f775385c3fcc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:36.086825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d13f4f-1496-478d-a0e2-f775385c3fcc None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:36.087517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d13f4f-1496-478d-a0e2-f775385c3fcc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:36.087823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d13f4f-1496-478d-a0e2-f775385c3fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:36.089230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d13f4f-1496-478d-a0e2-f775385c3fcc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:36.109865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 211/426] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:36.116087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d13f4f-1496-478d-a0e2-f775385c3fcc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:36.116766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de88171-3b4f-4f70-b6bf-8c431e6a7a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:36.117409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de88171-3b4f-4f70-b6bf-8c431e6a7a61 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:36.117691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de88171-3b4f-4f70-b6bf-8c431e6a7a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:36.117961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de88171-3b4f-4f70-b6bf-8c431e6a7a61 None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:36.119090 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de88171-3b4f-4f70-b6bf-8c431e6a7a61 None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:36.120581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de88171-3b4f-4f70-b6bf-8c431e6a7a61 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:36.122386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 216/427] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:35 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:36.128188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de88171-3b4f-4f70-b6bf-8c431e6a7a61 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:36.137984 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:36.193551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:36.280025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d13f4f-1496-478d-a0e2-f775385c3fcc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:36.293282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de88171-3b4f-4f70-b6bf-8c431e6a7a61 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:36.332023 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa50b8e-618c-446b-858a-7a8d1d41b0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:36.332864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa50b8e-618c-446b-858a-7a8d1d41b0fe None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:36.333150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa50b8e-618c-446b-858a-7a8d1d41b0fe None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:36.333420 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa50b8e-618c-446b-858a-7a8d1d41b0fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:36.337089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ccd4f-2791-4a7c-b52c-ca25c4587331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:36.338723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ccd4f-2791-4a7c-b52c-ca25c4587331 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:36.339271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ccd4f-2791-4a7c-b52c-ca25c4587331 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:36.339760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532ccd4f-2791-4a7c-b52c-ca25c4587331 None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:36.341275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ccd4f-2791-4a7c-b52c-ca25c4587331 None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:36.343320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532ccd4f-2791-4a7c-b52c-ca25c4587331 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:36.346035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 217/428] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:36 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:36.347008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ccd4f-2791-4a7c-b52c-ca25c4587331 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:36.420871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa50b8e-618c-446b-858a-7a8d1d41b0fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:36.423253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa50b8e-618c-446b-858a-7a8d1d41b0fe None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:36.432662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:36.470752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532ccd4f-2791-4a7c-b52c-ca25c4587331 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:36.482937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:36.511144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d1e5e-e809-4c4a-824e-4500e862b06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:36.511979 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d1e5e-e809-4c4a-824e-4500e862b06e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:36.512356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d1e5e-e809-4c4a-824e-4500e862b06e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:36.512728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d1e5e-e809-4c4a-824e-4500e862b06e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:36.524169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d1e5e-e809-4c4a-824e-4500e862b06e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-aa3d1e5e-e809-4c4a-824e-4500e862b06e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:36.531110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:36.532075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:36.532848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 218/429] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:36.533485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d1e5e-e809-4c4a-824e-4500e862b06e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:36.546272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 212/430] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 14:00:36.547481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa50b8e-618c-446b-858a-7a8d1d41b0fe None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:36.648168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d1e5e-e809-4c4a-824e-4500e862b06e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:36.651101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa50b8e-618c-446b-858a-7a8d1d41b0fe None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:36.680407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:36.687019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c6e87-9a14-4e71-90c8-cba8e2ae3187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:36.687862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c6e87-9a14-4e71-90c8-cba8e2ae3187 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:36.688185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da160da-9125-4cf4-b813-1601055baae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:36.688302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c6e87-9a14-4e71-90c8-cba8e2ae3187 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:36.688545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c6e87-9a14-4e71-90c8-cba8e2ae3187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:36.689278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c6e87-9a14-4e71-90c8-cba8e2ae3187 None admin] RBAC: Adding query filter params (group.id=fd088c52d4e249a69f4917e21f9ce0c5, role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:36.689502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da160da-9125-4cf4-b813-1601055baae6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:36.689598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c6e87-9a14-4e71-90c8-cba8e2ae3187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:36.690044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da160da-9125-4cf4-b813-1601055baae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:36.690500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da160da-9125-4cf4-b813-1601055baae6 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:36.691010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c6e87-9a14-4e71-90c8-cba8e2ae3187 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:36.691579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da160da-9125-4cf4-b813-1601055baae6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:36.692088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da160da-9125-4cf4-b813-1601055baae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:36.693960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da160da-9125-4cf4-b813-1601055baae6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:36.703027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 213/431] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:00:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:36.703571 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 219/432] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 10 14:00:36 2022] GET /identity/v3/role_assignments?group.id=fd088c52d4e249a69f4917e21f9ce0c5&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 611 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:36.704490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da160da-9125-4cf4-b813-1601055baae6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:36.719245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:36.720653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c6e87-9a14-4e71-90c8-cba8e2ae3187 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:36.835304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da160da-9125-4cf4-b813-1601055baae6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:36.871605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c6e87-9a14-4e71-90c8-cba8e2ae3187 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:36.907308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbc1a0-07c8-40b0-bb48-63fd42013b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:36.907981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbc1a0-07c8-40b0-bb48-63fd42013b4d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:36.908311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbc1a0-07c8-40b0-bb48-63fd42013b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:36.908629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fbc1a0-07c8-40b0-bb48-63fd42013b4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:36.909044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03377d89-e7dd-45c5-ae8d-91631fda8c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:36.909914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377d89-e7dd-45c5-ae8d-91631fda8c9b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:36.910378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377d89-e7dd-45c5-ae8d-91631fda8c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:36.910861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377d89-e7dd-45c5-ae8d-91631fda8c9b None admin] PATH_INFO: `/v3/projects/2c95a63dd5554fe1b35750e8ec50d808` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:36.912695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03377d89-e7dd-45c5-ae8d-91631fda8c9b None admin] RBAC: Authorizing `identity:get_project(project_id=2c95a63dd5554fe1b35750e8ec50d808)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:36.914386 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03377d89-e7dd-45c5-ae8d-91631fda8c9b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:36.916245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 214/433] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:36 2022] GET /identity/v3/projects/2c95a63dd5554fe1b35750e8ec50d808 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:36.922195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03377d89-e7dd-45c5-ae8d-91631fda8c9b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:36.986531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:36.997060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbc1a0-07c8-40b0-bb48-63fd42013b4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:36.998335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fbc1a0-07c8-40b0-bb48-63fd42013b4d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:37.033071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:37.058951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03377d89-e7dd-45c5-ae8d-91631fda8c9b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:37.100530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4fac2-cf66-4fb6-af11-12050a1deade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:37.101225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4fac2-cf66-4fb6-af11-12050a1deade None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:37.101597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4fac2-cf66-4fb6-af11-12050a1deade None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:37.101988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f4fac2-cf66-4fb6-af11-12050a1deade None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:37.103211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4fac2-cf66-4fb6-af11-12050a1deade None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:37.104821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f4fac2-cf66-4fb6-af11-12050a1deade None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:37.106791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 215/434] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:36 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:37.112934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4fac2-cf66-4fb6-af11-12050a1deade None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:37.174358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 220/435] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 14:00:37.180674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbc1a0-07c8-40b0-bb48-63fd42013b4d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:37.269441 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f4fac2-cf66-4fb6-af11-12050a1deade None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:37.300431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:37.308948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fbc1a0-07c8-40b0-bb48-63fd42013b4d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:37.309752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed00365-4116-4cf3-b35d-4da7424d2c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:37.310357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed00365-4116-4cf3-b35d-4da7424d2c48 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:37.310632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed00365-4116-4cf3-b35d-4da7424d2c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:37.310897 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed00365-4116-4cf3-b35d-4da7424d2c48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:37.311660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed00365-4116-4cf3-b35d-4da7424d2c48 None admin] RBAC: Adding query filter params (group.id=fd088c52d4e249a69f4917e21f9ce0c5, role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf, scope.project.id=2c95a63dd5554fe1b35750e8ec50d808) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:37.312115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed00365-4116-4cf3-b35d-4da7424d2c48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:37.313818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed00365-4116-4cf3-b35d-4da7424d2c48 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:37.321561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 216/436] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 10 14:00:37 2022] GET /identity/v3/role_assignments?group.id=fd088c52d4e249a69f4917e21f9ce0c5&scope.project.id=2c95a63dd5554fe1b35750e8ec50d808&role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:37.355724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206846e-0797-4d3e-8b9a-9c6707091e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:37.356613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206846e-0797-4d3e-8b9a-9c6707091e80 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:37.357055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206846e-0797-4d3e-8b9a-9c6707091e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:37.357470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206846e-0797-4d3e-8b9a-9c6707091e80 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:37.358949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206846e-0797-4d3e-8b9a-9c6707091e80 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:37.360673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206846e-0797-4d3e-8b9a-9c6707091e80 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:37.362909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 221/437] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:37 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:37.369322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed00365-4116-4cf3-b35d-4da7424d2c48 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:37.386364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:37.495512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed00365-4116-4cf3-b35d-4da7424d2c48 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:37.537010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6978da50-0aa6-4d44-8d15-39abcdc4fad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:37.537896 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978da50-0aa6-4d44-8d15-39abcdc4fad7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:37.538173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978da50-0aa6-4d44-8d15-39abcdc4fad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:37.538445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6978da50-0aa6-4d44-8d15-39abcdc4fad7 None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:37.539623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978da50-0aa6-4d44-8d15-39abcdc4fad7 None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:37.541083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6978da50-0aa6-4d44-8d15-39abcdc4fad7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:37.542946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 217/438] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:37 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:37.548762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206846e-0797-4d3e-8b9a-9c6707091e80 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:37.599494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288c945-6514-43eb-8059-e759ae56c915 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:37.599804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288c945-6514-43eb-8059-e759ae56c915 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:37.600032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288c945-6514-43eb-8059-e759ae56c915 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:37.601617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 218/439] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:00:37.606888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3d3ea-7d6a-498f-82b0-e47a2512ecb3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:37.607170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3d3ea-7d6a-498f-82b0-e47a2512ecb3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:37.607566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c3d3ea-7d6a-498f-82b0-e47a2512ecb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:37.635067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b6c3d3ea-7d6a-498f-82b0-e47a2512ecb3 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:37.636591 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b6c3d3ea-7d6a-498f-82b0-e47a2512ecb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:37.674179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206846e-0797-4d3e-8b9a-9c6707091e80 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:37.731841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 219/440] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:37.732251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dc39d-af07-43a5-9d60-e1ecfcb33b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:37.732963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dc39d-af07-43a5-9d60-e1ecfcb33b36 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:37.733234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dc39d-af07-43a5-9d60-e1ecfcb33b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:37.733501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dc39d-af07-43a5-9d60-e1ecfcb33b36 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:37.734289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dc39d-af07-43a5-9d60-e1ecfcb33b36 None admin] RBAC: Adding query filter params (group.id=fd088c52d4e249a69f4917e21f9ce0c5, role.id=bb37d85c981a4f79b9602ba7d0bc6fa5, scope.project.id=87ea676d839141b4b3e827be73b4e648) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:37.734596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dc39d-af07-43a5-9d60-e1ecfcb33b36 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:37.736427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dc39d-af07-43a5-9d60-e1ecfcb33b36 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:37.745542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 222/441] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 10 14:00:37 2022] GET /identity/v3/role_assignments?group.id=fd088c52d4e249a69f4917e21f9ce0c5&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:37.833263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84473913-2f2e-47d6-ac02-ea550868e9f0 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:37.833546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84473913-2f2e-47d6-ac02-ea550868e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:37.833822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84473913-2f2e-47d6-ac02-ea550868e9f0 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:37.835390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 220/442] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:37.841210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc48e5-0d4f-4ef8-82a4-ed744dbe4473 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:37.841526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc48e5-0d4f-4ef8-82a4-ed744dbe4473 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:37.841766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc48e5-0d4f-4ef8-82a4-ed744dbe4473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:37.863304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-65dc48e5-0d4f-4ef8-82a4-ed744dbe4473 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:37.864814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-65dc48e5-0d4f-4ef8-82a4-ed744dbe4473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:37.893541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:37.945530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 223/443] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:38.016128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84473913-2f2e-47d6-ac02-ea550868e9f0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:38.027844 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:38.126413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84473913-2f2e-47d6-ac02-ea550868e9f0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:38.162391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac767021-af74-4e7c-bd7f-a40c362f4a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:38.163015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac767021-af74-4e7c-bd7f-a40c362f4a6e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:38.163289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac767021-af74-4e7c-bd7f-a40c362f4a6e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:38.164070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac767021-af74-4e7c-bd7f-a40c362f4a6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:38.176211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac767021-af74-4e7c-bd7f-a40c362f4a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-ac767021-af74-4e7c-bd7f-a40c362f4a6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:38.181559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:38.182457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:38.183256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 221/444] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:38.188581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc48e5-0d4f-4ef8-82a4-ed744dbe4473 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:38.217018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac767021-af74-4e7c-bd7f-a40c362f4a6e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:38.283884 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:38.288549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc48e5-0d4f-4ef8-82a4-ed744dbe4473 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:38.318300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac767021-af74-4e7c-bd7f-a40c362f4a6e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:38.322033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b92b2f-3ddd-40b8-a408-0cf45d336a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:38.322633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b92b2f-3ddd-40b8-a408-0cf45d336a3d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:38.322903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b92b2f-3ddd-40b8-a408-0cf45d336a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:38.323167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b92b2f-3ddd-40b8-a408-0cf45d336a3d None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:38.323858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b92b2f-3ddd-40b8-a408-0cf45d336a3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:38.324156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b92b2f-3ddd-40b8-a408-0cf45d336a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:38.325510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b92b2f-3ddd-40b8-a408-0cf45d336a3d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:38.333565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 224/445] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:38.339208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b92b2f-3ddd-40b8-a408-0cf45d336a3d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:38.353574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd31e1-292c-425c-8605-2e0ec898dea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:38.354185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd31e1-292c-425c-8605-2e0ec898dea7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:38.354463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd31e1-292c-425c-8605-2e0ec898dea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:38.354743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccd31e1-292c-425c-8605-2e0ec898dea7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:38.360434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:38.365776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccd31e1-292c-425c-8605-2e0ec898dea7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-7ccd31e1-292c-425c-8605-2e0ec898dea7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:38.370474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:38.371566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:38.372585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 222/446] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:00:38.377365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd31e1-292c-425c-8605-2e0ec898dea7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:38.449994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b92b2f-3ddd-40b8-a408-0cf45d336a3d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:38.479414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccd31e1-292c-425c-8605-2e0ec898dea7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:38.484474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527a94-a297-411d-b3bb-371b1cbfc920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:38.485099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527a94-a297-411d-b3bb-371b1cbfc920 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:38.485371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527a94-a297-411d-b3bb-371b1cbfc920 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:38.485636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f527a94-a297-411d-b3bb-371b1cbfc920 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:38.506125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:38.514186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07998a-8c58-44a1-917e-3736827d0318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:38.514777 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07998a-8c58-44a1-917e-3736827d0318 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:38.515052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07998a-8c58-44a1-917e-3736827d0318 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:38.515318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db07998a-8c58-44a1-917e-3736827d0318 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:38.516005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07998a-8c58-44a1-917e-3736827d0318 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:38.516305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07998a-8c58-44a1-917e-3736827d0318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:38.517689 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db07998a-8c58-44a1-917e-3736827d0318 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:38.525684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 223/447] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:38.531402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07998a-8c58-44a1-917e-3736827d0318 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:38.553126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f527a94-a297-411d-b3bb-371b1cbfc920 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:38.554581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f527a94-a297-411d-b3bb-371b1cbfc920 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:38.575339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:38.629392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db07998a-8c58-44a1-917e-3736827d0318 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:38.652382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 225/448] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 14:00:38.657933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527a94-a297-411d-b3bb-371b1cbfc920 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:38.665734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e49f5-ad91-4991-9fd9-74fc69aef0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:38.666336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e49f5-ad91-4991-9fd9-74fc69aef0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:38.666608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e49f5-ad91-4991-9fd9-74fc69aef0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:38.666875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285e49f5-ad91-4991-9fd9-74fc69aef0b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:38.722303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:38.733437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e49f5-ad91-4991-9fd9-74fc69aef0b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:38.734891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285e49f5-ad91-4991-9fd9-74fc69aef0b5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:38.756562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f527a94-a297-411d-b3bb-371b1cbfc920 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:38.776356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:38.790752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc9a80-4e03-4402-8006-8fb78dc5e52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:38.791371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc9a80-4e03-4402-8006-8fb78dc5e52a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:38.791683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc9a80-4e03-4402-8006-8fb78dc5e52a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:38.791956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc9a80-4e03-4402-8006-8fb78dc5e52a None admin] PATH_INFO: `/v3/projects/7a00398c85824c23b37ffe5fee90b840` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:38.793072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc9a80-4e03-4402-8006-8fb78dc5e52a None admin] RBAC: Authorizing `identity:get_project(project_id=7a00398c85824c23b37ffe5fee90b840)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:38.794493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc9a80-4e03-4402-8006-8fb78dc5e52a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:38.796132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 226/449] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/projects/7a00398c85824c23b37ffe5fee90b840 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:38.802120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc9a80-4e03-4402-8006-8fb78dc5e52a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:38.834637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 224/450] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 10 14:00:38.840183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e49f5-ad91-4991-9fd9-74fc69aef0b5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:38.904535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc9a80-4e03-4402-8006-8fb78dc5e52a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:38.927679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:38.936938 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a0174-4dbc-4a1a-abfa-90f68cebd0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:38.937466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285e49f5-ad91-4991-9fd9-74fc69aef0b5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:38.937583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a0174-4dbc-4a1a-abfa-90f68cebd0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:38.937826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a0174-4dbc-4a1a-abfa-90f68cebd0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:38.938099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a0174-4dbc-4a1a-abfa-90f68cebd0a5 None admin] PATH_INFO: `/v3/groups/50a50b766d144d65a355c672d440a33e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:38.939196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a0174-4dbc-4a1a-abfa-90f68cebd0a5 None admin] RBAC: Authorizing `identity:get_group(group_id=50a50b766d144d65a355c672d440a33e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:38.940695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a0174-4dbc-4a1a-abfa-90f68cebd0a5 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:38.942424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 227/451] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/groups/50a50b766d144d65a355c672d440a33e => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:38.948126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a0174-4dbc-4a1a-abfa-90f68cebd0a5 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:38.971849 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8576464-72ed-414f-b62c-f0da8de67310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:38.972466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8576464-72ed-414f-b62c-f0da8de67310 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:38.972740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8576464-72ed-414f-b62c-f0da8de67310 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:38.973011 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8576464-72ed-414f-b62c-f0da8de67310 None admin] PATH_INFO: `/v3/projects/87ea676d839141b4b3e827be73b4e648` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:38.974386 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8576464-72ed-414f-b62c-f0da8de67310 None admin] RBAC: Authorizing `identity:get_project(project_id=87ea676d839141b4b3e827be73b4e648)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:38.975856 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8576464-72ed-414f-b62c-f0da8de67310 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:38.977468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 225/452] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/projects/87ea676d839141b4b3e827be73b4e648 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:38.982872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8576464-72ed-414f-b62c-f0da8de67310 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:39.003230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:39.046117 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a0174-4dbc-4a1a-abfa-90f68cebd0a5 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:39.079142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029af78-5483-42ef-8a23-97be820edecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:39.079770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029af78-5483-42ef-8a23-97be820edecc None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:39.079873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8576464-72ed-414f-b62c-f0da8de67310 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:39.080045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029af78-5483-42ef-8a23-97be820edecc None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:39.080312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0029af78-5483-42ef-8a23-97be820edecc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:39.081024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029af78-5483-42ef-8a23-97be820edecc None admin] RBAC: Adding query filter params (group.id=50a50b766d144d65a355c672d440a33e, role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=7a00398c85824c23b37ffe5fee90b840) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:39.081335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029af78-5483-42ef-8a23-97be820edecc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:39.082962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0029af78-5483-42ef-8a23-97be820edecc None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:39.088567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 228/453] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/role_assignments?group.id=50a50b766d144d65a355c672d440a33e&scope.project.id=7a00398c85824c23b37ffe5fee90b840&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:39.120761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7910d8d-5c2e-4804-a792-81b619b8cf6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:39.121401 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7910d8d-5c2e-4804-a792-81b619b8cf6c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:39.121672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7910d8d-5c2e-4804-a792-81b619b8cf6c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:39.121937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7910d8d-5c2e-4804-a792-81b619b8cf6c None admin] PATH_INFO: `/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:39.123095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7910d8d-5c2e-4804-a792-81b619b8cf6c None admin] RBAC: Authorizing `identity:get_group(group_id=fd088c52d4e249a69f4917e21f9ce0c5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:39.124624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7910d8d-5c2e-4804-a792-81b619b8cf6c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:39.126264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 226/454] 10.222.0.26 () {56 vars in 1180 bytes} [Sat Sep 10 14:00:38 2022] GET /identity/v3/groups/fd088c52d4e249a69f4917e21f9ce0c5 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:00:39.133109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029af78-5483-42ef-8a23-97be820edecc None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:39.237406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0029af78-5483-42ef-8a23-97be820edecc None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:39.258018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:39.273415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1902e0-f54e-41a0-82ca-25b1eb19eeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:39.274019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1902e0-f54e-41a0-82ca-25b1eb19eeaa None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:39.274293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1902e0-f54e-41a0-82ca-25b1eb19eeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:39.274561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1902e0-f54e-41a0-82ca-25b1eb19eeaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:39.275282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1902e0-f54e-41a0-82ca-25b1eb19eeaa None admin] RBAC: Adding query filter params (group.id=fd088c52d4e249a69f4917e21f9ce0c5, role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf, scope.project.id=87ea676d839141b4b3e827be73b4e648) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:39.275677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1902e0-f54e-41a0-82ca-25b1eb19eeaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:39.277101 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1902e0-f54e-41a0-82ca-25b1eb19eeaa None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:39.282906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 229/455] 10.222.0.26 () {56 vars in 1376 bytes} [Sat Sep 10 14:00:39 2022] GET /identity/v3/role_assignments?group.id=fd088c52d4e249a69f4917e21f9ce0c5&scope.project.id=87ea676d839141b4b3e827be73b4e648&role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:42.114903 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94424dd7-d35d-4b3e-a24d-23288d0880d9 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:42.115569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94424dd7-d35d-4b3e-a24d-23288d0880d9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:42.115790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94424dd7-d35d-4b3e-a24d-23288d0880d9 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:42.117598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 227/456] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:42.122677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03f576-9f06-4a61-8833-0b410b5ccd32 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:42.123419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03f576-9f06-4a61-8833-0b410b5ccd32 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:42.123884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f03f576-9f06-4a61-8833-0b410b5ccd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:42.148403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-3f03f576-9f06-4a61-8833-0b410b5ccd32 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:42.149817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3f03f576-9f06-4a61-8833-0b410b5ccd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:42.222951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0152ab-658f-484c-a23e-57d6aaffd9ea None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:42.223262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0152ab-658f-484c-a23e-57d6aaffd9ea None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:42.224087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0152ab-658f-484c-a23e-57d6aaffd9ea None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:42.225755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 228/457] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:42.232148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cda2b-b2cc-40ac-a54e-c04fcf86c34e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:42.232706 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cda2b-b2cc-40ac-a54e-c04fcf86c34e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:42.233049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cda2b-b2cc-40ac-a54e-c04fcf86c34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:42.243373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 230/458] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:42.264503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e59cda2b-b2cc-40ac-a54e-c04fcf86c34e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:42.267266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e59cda2b-b2cc-40ac-a54e-c04fcf86c34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:42.297830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:42.366695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 229/459] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:42.459874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbcba6-c02d-46b3-8f0f-aa1942fcea8d None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:42.460305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbcba6-c02d-46b3-8f0f-aa1942fcea8d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:42.460676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcbcba6-c02d-46b3-8f0f-aa1942fcea8d None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:42.462286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 231/460] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:42.477268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5f89f-9811-44d0-89d7-ef94aadc3f24 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:42.477801 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5f89f-9811-44d0-89d7-ef94aadc3f24 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:42.478299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5f89f-9811-44d0-89d7-ef94aadc3f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:42.503601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-81c5f89f-9811-44d0-89d7-ef94aadc3f24 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:42.505606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-81c5f89f-9811-44d0-89d7-ef94aadc3f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:42.601052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 230/461] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:42.672660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbcba6-c02d-46b3-8f0f-aa1942fcea8d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:42.740681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5f89f-9811-44d0-89d7-ef94aadc3f24 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:42.789132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcbcba6-c02d-46b3-8f0f-aa1942fcea8d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:42.804746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:42.826428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ed905-da5d-4d29-a05b-78e242296328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:42.827060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ed905-da5d-4d29-a05b-78e242296328 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:42.827334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ed905-da5d-4d29-a05b-78e242296328 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:42.827636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ed905-da5d-4d29-a05b-78e242296328 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:42.839168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ed905-da5d-4d29-a05b-78e242296328 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:42.840580 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:42.841894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ed905-da5d-4d29-a05b-78e242296328 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:42.856836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5f89f-9811-44d0-89d7-ef94aadc3f24 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-c04ed905-da5d-4d29-a05b-78e242296328 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:42.859220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:42.860121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:42.860974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:42.860974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:42.860974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:42.860974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 232/462] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:42.866843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a3d24-47fb-4e62-9299-1d6d7a3e3d2d None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:42.866843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a3d24-47fb-4e62-9299-1d6d7a3e3d2d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:42.866843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866a3d24-47fb-4e62-9299-1d6d7a3e3d2d None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:42.866843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 233/463] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:42.867819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a3d24-47fb-4e62-9299-1d6d7a3e3d2d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:42.899121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55c364-757c-4eb3-be64-df4672e8c5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:42.899786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55c364-757c-4eb3-be64-df4672e8c5ac None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:42.900061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55c364-757c-4eb3-be64-df4672e8c5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:42.900332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55c364-757c-4eb3-be64-df4672e8c5ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:42.911376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55c364-757c-4eb3-be64-df4672e8c5ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:42.912986 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55c364-757c-4eb3-be64-df4672e8c5ac None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-ee55c364-757c-4eb3-be64-df4672e8c5ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:42.929458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:42.931579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:42.933643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:42.933643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:42.933643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:42.933643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 231/464] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:42.933643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b76230-ec1e-4216-8745-76c2cc08dd7d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:42.933643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b76230-ec1e-4216-8745-76c2cc08dd7d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:42.933643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b76230-ec1e-4216-8745-76c2cc08dd7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:42.952587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c5b76230-ec1e-4216-8745-76c2cc08dd7d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:42.953878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c5b76230-ec1e-4216-8745-76c2cc08dd7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:42.977417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866a3d24-47fb-4e62-9299-1d6d7a3e3d2d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:43.016112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cd87c-0b78-4d48-973e-9298cf7127e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:43.016867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cd87c-0b78-4d48-973e-9298cf7127e2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.017143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cd87c-0b78-4d48-973e-9298cf7127e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.017415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24cd87c-0b78-4d48-973e-9298cf7127e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.018049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cd87c-0b78-4d48-973e-9298cf7127e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:43.018348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cd87c-0b78-4d48-973e-9298cf7127e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:43.019789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24cd87c-0b78-4d48-973e-9298cf7127e2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:43.026494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 232/465] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:43.035575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 234/466] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:43.035575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cd87c-0b78-4d48-973e-9298cf7127e2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:43.037113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b76230-ec1e-4216-8745-76c2cc08dd7d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:43.065389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:43.125969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:43.144279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24cd87c-0b78-4d48-973e-9298cf7127e2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:43.170882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b76230-ec1e-4216-8745-76c2cc08dd7d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:43.180775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20ce8a-2797-4d3b-acc7-8c8db9e9209e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:43.181665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20ce8a-2797-4d3b-acc7-8c8db9e9209e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.182083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20ce8a-2797-4d3b-acc7-8c8db9e9209e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.182519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20ce8a-2797-4d3b-acc7-8c8db9e9209e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.195151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20ce8a-2797-4d3b-acc7-8c8db9e9209e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:43.196969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20ce8a-2797-4d3b-acc7-8c8db9e9209e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:43.212844 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01186e-5511-49f5-990f-0175339c6c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:43.213575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01186e-5511-49f5-990f-0175339c6c1b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-4c20ce8a-2797-4d3b-acc7-8c8db9e9209e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:43.213733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:43.214709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:43.215574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:43.215574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:43.215574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:43.215731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01186e-5511-49f5-990f-0175339c6c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.215731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01186e-5511-49f5-990f-0175339c6c1b None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.215731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01186e-5511-49f5-990f-0175339c6c1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:43.216606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01186e-5511-49f5-990f-0175339c6c1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:43.217900 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 235/467] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:43.218941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20ce8a-2797-4d3b-acc7-8c8db9e9209e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:43.223478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e01186e-5511-49f5-990f-0175339c6c1b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:43.243586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 233/468] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:43.244760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01186e-5511-49f5-990f-0175339c6c1b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:43.323896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:43.348605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20ce8a-2797-4d3b-acc7-8c8db9e9209e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:43.372536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01186e-5511-49f5-990f-0175339c6c1b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:43.386455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9b5bf-4ffc-44e3-8079-db59da44669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:43.387060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9b5bf-4ffc-44e3-8079-db59da44669b None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.387340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9b5bf-4ffc-44e3-8079-db59da44669b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.387636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e9b5bf-4ffc-44e3-8079-db59da44669b None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.388283 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9b5bf-4ffc-44e3-8079-db59da44669b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:43.389712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e9b5bf-4ffc-44e3-8079-db59da44669b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:43.403222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:43.413278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 236/469] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:00:43 2022] POST /identity/v3/users => generated 246 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:00:43.413499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d1b4a-5e30-46a9-9489-79df448ec8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:43.414159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d1b4a-5e30-46a9-9489-79df448ec8c0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.414442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d1b4a-5e30-46a9-9489-79df448ec8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.414709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0d1b4a-5e30-46a9-9489-79df448ec8c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.415120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9b5bf-4ffc-44e3-8079-db59da44669b None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:43.415357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d1b4a-5e30-46a9-9489-79df448ec8c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:43.415735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d1b4a-5e30-46a9-9489-79df448ec8c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:43.417158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0d1b4a-5e30-46a9-9489-79df448ec8c0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:43.449592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 234/470] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:43.449592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d1b4a-5e30-46a9-9489-79df448ec8c0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:43.554475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e9b5bf-4ffc-44e3-8079-db59da44669b None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:43.573785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0d1b4a-5e30-46a9-9489-79df448ec8c0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:43.599789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d33c2-8a96-4d9f-a82a-bf47e43b48f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:43.600782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d33c2-8a96-4d9f-a82a-bf47e43b48f7 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.601201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d33c2-8a96-4d9f-a82a-bf47e43b48f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.601613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4d33c2-8a96-4d9f-a82a-bf47e43b48f7 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.602410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d33c2-8a96-4d9f-a82a-bf47e43b48f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:43.604097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4d33c2-8a96-4d9f-a82a-bf47e43b48f7 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:43.617935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:43.618832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea661b-8f7b-42f3-9885-e3436a1ac934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:43.619491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea661b-8f7b-42f3-9885-e3436a1ac934 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.619862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea661b-8f7b-42f3-9885-e3436a1ac934 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.620131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea661b-8f7b-42f3-9885-e3436a1ac934 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.633785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea661b-8f7b-42f3-9885-e3436a1ac934 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:43.635366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea661b-8f7b-42f3-9885-e3436a1ac934 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:43.643753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 237/471] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:00:43 2022] POST /identity/v3/users => generated 248 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:00:43.644767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d33c2-8a96-4d9f-a82a-bf47e43b48f7 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-1fea661b-8f7b-42f3-9885-e3436a1ac934 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:43.653592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:43.654565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:43.655462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:43.655462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:43.655462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:43.655462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 235/472] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:43.661921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea661b-8f7b-42f3-9885-e3436a1ac934 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:43.725558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:43.767899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4d33c2-8a96-4d9f-a82a-bf47e43b48f7 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:43.791582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea661b-8f7b-42f3-9885-e3436a1ac934 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:43.806889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c0f14-2fb2-4dfb-93bd-a107a10e067b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:43.807531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c0f14-2fb2-4dfb-93bd-a107a10e067b None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.807827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c0f14-2fb2-4dfb-93bd-a107a10e067b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.808083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c0f14-2fb2-4dfb-93bd-a107a10e067b None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.808743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c0f14-2fb2-4dfb-93bd-a107a10e067b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:43.810211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8c0f14-2fb2-4dfb-93bd-a107a10e067b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:43.835231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72452c-eb67-4353-9b0a-9aca1c096641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:43.836139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72452c-eb67-4353-9b0a-9aca1c096641 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.836595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72452c-eb67-4353-9b0a-9aca1c096641 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.836700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72452c-eb67-4353-9b0a-9aca1c096641 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.837317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72452c-eb67-4353-9b0a-9aca1c096641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:43.837617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72452c-eb67-4353-9b0a-9aca1c096641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:43.837985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 238/473] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:00:43 2022] POST /identity/v3/users => generated 248 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:00:43.839074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72452c-eb67-4353-9b0a-9aca1c096641 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:43.839304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ba1d5-f024-4348-b67d-5d47d4d976e3 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.839556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ba1d5-f024-4348-b67d-5d47d4d976e3 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.839817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ba1d5-f024-4348-b67d-5d47d4d976e3 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.841521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 239/474] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:43.849117 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4f5ae-ad46-4789-a79d-676334682da4 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:43.849542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4f5ae-ad46-4789-a79d-676334682da4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:43.850020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4f5ae-ad46-4789-a79d-676334682da4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:43.856371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 236/475] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:43.862196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72452c-eb67-4353-9b0a-9aca1c096641 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:43.882043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-38a4f5ae-ad46-4789-a79d-676334682da4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:43.883255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-38a4f5ae-ad46-4789-a79d-676334682da4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:43.953449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:44.000502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 240/476] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:44.001644 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:44.009391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72452c-eb67-4353-9b0a-9aca1c096641 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:44.049778 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe0f3-5023-4b88-a0c2-154729c03691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:44.050677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe0f3-5023-4b88-a0c2-154729c03691 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:44.051108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe0f3-5023-4b88-a0c2-154729c03691 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:44.051637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe0f3-5023-4b88-a0c2-154729c03691 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:44.052906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fe0f3-5023-4b88-a0c2-154729c03691 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:44.054602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1fe0f3-5023-4b88-a0c2-154729c03691 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:44.082582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 237/477] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:00:43 2022] POST /identity/v3/users => generated 249 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:00:44.292861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4f5ae-ad46-4789-a79d-676334682da4 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:44.419100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4f5ae-ad46-4789-a79d-676334682da4 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:44.476262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d4410-7b35-4384-baa0-5648fa619824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:44.477233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d4410-7b35-4384-baa0-5648fa619824 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:44.477733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d4410-7b35-4384-baa0-5648fa619824 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:44.478212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5d4410-7b35-4384-baa0-5648fa619824 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:44.479091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d4410-7b35-4384-baa0-5648fa619824 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:44.480676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5d4410-7b35-4384-baa0-5648fa619824 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:44.490818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.sql.core [None req-fb5d4410-7b35-4384-baa0-5648fa619824 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 14:00:44.490818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 10 14:00:44.490818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [parameters: {'id': '4dcb70b9b46a4d8bac4a6e7631d3987a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 10 14:00:44.490818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81588) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-fb5d4410-7b35-4384-baa0-5648fa619824 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application conn.query(q) Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result.read() Sep 10 14:00:44.512870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return ref.to_dict() Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 14:00:44.513823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application next(self.gen) Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application yield resource Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application next(self.gen) Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise self.value Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application session.commit() Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 14:00:44.514695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.session.flush() Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application compat.raise_( Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 14:00:44.515533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 14:00:44.517156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application conn.query(q) Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result.read() Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 14:00:44.517946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [parameters: {'id': '4dcb70b9b46a4d8bac4a6e7631d3987a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:44.518718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 10 14:00:44.519512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:44.524028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 241/478] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Sep 10 14:00:44 2022] POST /identity/v3/roles => generated 144 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:44.529861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe0f3-5023-4b88-a0c2-154729c03691 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:44.641385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe0f3-5023-4b88-a0c2-154729c03691 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:44.681077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc9f5d-20d1-4265-9d6f-d1dfb9f8c1fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:44.681941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc9f5d-20d1-4265-9d6f-d1dfb9f8c1fb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:44.682383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc9f5d-20d1-4265-9d6f-d1dfb9f8c1fb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:44.682792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc9f5d-20d1-4265-9d6f-d1dfb9f8c1fb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:44.694357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc9f5d-20d1-4265-9d6f-d1dfb9f8c1fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-b7fc9f5d-20d1-4265-9d6f-d1dfb9f8c1fb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:44.700823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:44.701874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 10 14:00:44.702839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:44.705103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 238/479] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:00:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:44.711352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d4410-7b35-4384-baa0-5648fa619824 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:44.758588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:44.821632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5d4410-7b35-4384-baa0-5648fa619824 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:44.857496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d1940e-26e0-4fb2-ae36-a08a4b1c25d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:44.858365 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1940e-26e0-4fb2-ae36-a08a4b1c25d0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:44.858857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1940e-26e0-4fb2-ae36-a08a4b1c25d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:44.859316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d1940e-26e0-4fb2-ae36-a08a4b1c25d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:44.860213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1940e-26e0-4fb2-ae36-a08a4b1c25d0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:44.860652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1940e-26e0-4fb2-ae36-a08a4b1c25d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:44.862154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d1940e-26e0-4fb2-ae36-a08a4b1c25d0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:44.871457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 242/480] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:46.351720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e628f79-10b5-44a6-b7d5-dc1808e1f1cb None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:46.352247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e628f79-10b5-44a6-b7d5-dc1808e1f1cb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:46.352247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e628f79-10b5-44a6-b7d5-dc1808e1f1cb None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:46.353930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 239/481] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:46.359752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411b331-74d7-4249-adc6-9118aa4251bf None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:46.360253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411b331-74d7-4249-adc6-9118aa4251bf None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:46.360323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9411b331-74d7-4249-adc6-9118aa4251bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:46.385738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-9411b331-74d7-4249-adc6-9118aa4251bf None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:46.386951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9411b331-74d7-4249-adc6-9118aa4251bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:46.482466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 243/482] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:46.498861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e3ab-0cf9-4e95-ab12-42d664fbc3e9 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:46.498861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e3ab-0cf9-4e95-ab12-42d664fbc3e9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:46.498861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760e3ab-0cf9-4e95-ab12-42d664fbc3e9 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:46.500212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 240/483] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:46.512942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab33473-eff1-4c9c-b855-a8b782a30ee1 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:46.513222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab33473-eff1-4c9c-b855-a8b782a30ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:46.513492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab33473-eff1-4c9c-b855-a8b782a30ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:46.540465 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2ab33473-eff1-4c9c-b855-a8b782a30ee1 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:46.541652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2ab33473-eff1-4c9c-b855-a8b782a30ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:46.585059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:46.633754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 244/484] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:46.844307 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0dff4-eb21-4aab-ba9d-8e3d5ed73770 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:46.844921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0dff4-eb21-4aab-ba9d-8e3d5ed73770 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:46.845115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0dff4-eb21-4aab-ba9d-8e3d5ed73770 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:46.846705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 241/485] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:46.852813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375cfb5-18ce-4a67-808d-6ff7db0797de None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:46.853105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375cfb5-18ce-4a67-808d-6ff7db0797de None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:46.853397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2375cfb5-18ce-4a67-808d-6ff7db0797de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:46.873003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0dff4-eb21-4aab-ba9d-8e3d5ed73770 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:46.875826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2375cfb5-18ce-4a67-808d-6ff7db0797de None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:46.877009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2375cfb5-18ce-4a67-808d-6ff7db0797de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:46.900685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:46.963260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 245/486] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:46.964355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375cfb5-18ce-4a67-808d-6ff7db0797de None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.010273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0dff4-eb21-4aab-ba9d-8e3d5ed73770 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.045503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076ae38-67b6-488a-9ac3-000243bc1a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.046121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076ae38-67b6-488a-9ac3-000243bc1a85 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.046390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076ae38-67b6-488a-9ac3-000243bc1a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.046655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076ae38-67b6-488a-9ac3-000243bc1a85 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.058318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076ae38-67b6-488a-9ac3-000243bc1a85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-5076ae38-67b6-488a-9ac3-000243bc1a85 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:47.063184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:47.064277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:47.065173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 242/487] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:47.065822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19684f-6225-430a-b0cf-5b5a0468d1af None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.066092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19684f-6225-430a-b0cf-5b5a0468d1af None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.066555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c19684f-6225-430a-b0cf-5b5a0468d1af None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.072371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 243/488] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:47.076834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19684f-6225-430a-b0cf-5b5a0468d1af None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.089726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:47.100523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2375cfb5-18ce-4a67-808d-6ff7db0797de None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.140285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e7ab76-a23e-4fbc-80cb-f082d4673c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.141263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7ab76-a23e-4fbc-80cb-f082d4673c97 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.141998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7ab76-a23e-4fbc-80cb-f082d4673c97 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.142945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e7ab76-a23e-4fbc-80cb-f082d4673c97 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.157856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e7ab76-a23e-4fbc-80cb-f082d4673c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-a8e7ab76-a23e-4fbc-80cb-f082d4673c97 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:47.164919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:47.165770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:47.166554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:47.173544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 246/489] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:47.173544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1e57b-e029-465e-b98c-0ff794559fc5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.173544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1e57b-e029-465e-b98c-0ff794559fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.174173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa1e57b-e029-465e-b98c-0ff794559fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.184376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:47.197033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2aa1e57b-e029-465e-b98c-0ff794559fc5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:47.199197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2aa1e57b-e029-465e-b98c-0ff794559fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:47.209517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c19684f-6225-430a-b0cf-5b5a0468d1af None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.245527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31252e84-87ba-4c81-b2a1-b34117f7b848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.246198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31252e84-87ba-4c81-b2a1-b34117f7b848 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.246475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31252e84-87ba-4c81-b2a1-b34117f7b848 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.246743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31252e84-87ba-4c81-b2a1-b34117f7b848 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.247404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31252e84-87ba-4c81-b2a1-b34117f7b848 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:47.247743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31252e84-87ba-4c81-b2a1-b34117f7b848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:47.249188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31252e84-87ba-4c81-b2a1-b34117f7b848 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:47.262827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 244/490] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:47.263725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31252e84-87ba-4c81-b2a1-b34117f7b848 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.278042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 247/491] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:47.278875 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1e57b-e029-465e-b98c-0ff794559fc5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.376091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:47.400064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa1e57b-e029-465e-b98c-0ff794559fc5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.404356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31252e84-87ba-4c81-b2a1-b34117f7b848 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.447927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ac09e-a3fc-4994-ade5-4e196f74b1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.448367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359dd08-ff09-4a67-81d8-dea4164d83e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.448578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ac09e-a3fc-4994-ade5-4e196f74b1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.448832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ac09e-a3fc-4994-ade5-4e196f74b1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.449099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ac09e-a3fc-4994-ade5-4e196f74b1f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.449948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359dd08-ff09-4a67-81d8-dea4164d83e8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.450437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359dd08-ff09-4a67-81d8-dea4164d83e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.450918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1359dd08-ff09-4a67-81d8-dea4164d83e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.451814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359dd08-ff09-4a67-81d8-dea4164d83e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:47.452323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359dd08-ff09-4a67-81d8-dea4164d83e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:47.453926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1359dd08-ff09-4a67-81d8-dea4164d83e8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:47.460639 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:47.461619 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546ac09e-a3fc-4994-ade5-4e196f74b1f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:47.465196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 245/492] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:47.466310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359dd08-ff09-4a67-81d8-dea4164d83e8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-546ac09e-a3fc-4994-ade5-4e196f74b1f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:47.468422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:47.469692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:47.470667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:47.473377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 248/493] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:47.474341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ac09e-a3fc-4994-ade5-4e196f74b1f1 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.593840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1359dd08-ff09-4a67-81d8-dea4164d83e8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.594664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546ac09e-a3fc-4994-ade5-4e196f74b1f1 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.632048 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb005e82-ec77-4a12-8558-1f15d01a3c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.632700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb005e82-ec77-4a12-8558-1f15d01a3c54 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.632975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb005e82-ec77-4a12-8558-1f15d01a3c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.633247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb005e82-ec77-4a12-8558-1f15d01a3c54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.634539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370e48c-cbd4-4992-b237-300e5cbe9b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.635188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370e48c-cbd4-4992-b237-300e5cbe9b94 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.635742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370e48c-cbd4-4992-b237-300e5cbe9b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.636020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370e48c-cbd4-4992-b237-300e5cbe9b94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.644475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb005e82-ec77-4a12-8558-1f15d01a3c54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:47.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb005e82-ec77-4a12-8558-1f15d01a3c54 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:47.650151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370e48c-cbd4-4992-b237-300e5cbe9b94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:47.651645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6370e48c-cbd4-4992-b237-300e5cbe9b94 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:47.653736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-eb005e82-ec77-4a12-8558-1f15d01a3c54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:47.662646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:47.663593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:47.665223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:47.665223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:47.665223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:47.665223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 246/494] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:47.665223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb005e82-ec77-4a12-8558-1f15d01a3c54 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-6370e48c-cbd4-4992-b237-300e5cbe9b94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:47.675637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:47.676851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:47.677744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:47.677744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:47.677744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:47.677744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 249/495] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:47.677971 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370e48c-cbd4-4992-b237-300e5cbe9b94 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.725520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:47.774749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb005e82-ec77-4a12-8558-1f15d01a3c54 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.783137 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370e48c-cbd4-4992-b237-300e5cbe9b94 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.807254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7403d16-cef7-468f-90ba-d22956c3ee61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.807862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7403d16-cef7-468f-90ba-d22956c3ee61 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.808139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7403d16-cef7-468f-90ba-d22956c3ee61 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.808412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7403d16-cef7-468f-90ba-d22956c3ee61 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.809058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7403d16-cef7-468f-90ba-d22956c3ee61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:47.809352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7403d16-cef7-468f-90ba-d22956c3ee61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:47.810722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7403d16-cef7-468f-90ba-d22956c3ee61 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:47.815844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a0095-d3b5-4a9e-83ac-9ec40f480991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.816462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a0095-d3b5-4a9e-83ac-9ec40f480991 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.816736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a0095-d3b5-4a9e-83ac-9ec40f480991 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.817568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461a0095-d3b5-4a9e-83ac-9ec40f480991 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.819182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 247/496] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:47.819994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7403d16-cef7-468f-90ba-d22956c3ee61 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.832399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461a0095-d3b5-4a9e-83ac-9ec40f480991 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-461a0095-d3b5-4a9e-83ac-9ec40f480991 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:47.838896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:47.839791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:47.840550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 250/497] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:47.841172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a0095-d3b5-4a9e-83ac-9ec40f480991 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.894553 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:47.921394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7403d16-cef7-468f-90ba-d22956c3ee61 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.935020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:47.939989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461a0095-d3b5-4a9e-83ac-9ec40f480991 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:47.955298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e664e90-6fef-4c8b-be66-e55948c85353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.955919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e664e90-6fef-4c8b-be66-e55948c85353 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.956196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e664e90-6fef-4c8b-be66-e55948c85353 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.956463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e664e90-6fef-4c8b-be66-e55948c85353 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.957092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e664e90-6fef-4c8b-be66-e55948c85353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:47.957387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e664e90-6fef-4c8b-be66-e55948c85353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:47.958895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e664e90-6fef-4c8b-be66-e55948c85353 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:47.970661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 248/498] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:47.971649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a29230-5709-44e6-8898-de38299ea595 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.971928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a29230-5709-44e6-8898-de38299ea595 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.972192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a29230-5709-44e6-8898-de38299ea595 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.973418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 249/499] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:47.974096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a29230-5709-44e6-8898-de38299ea595 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:47.978900 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae2e9f-1379-46dc-9a88-be2bc9e98de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:47.979468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae2e9f-1379-46dc-9a88-be2bc9e98de8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:47.979758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae2e9f-1379-46dc-9a88-be2bc9e98de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:47.980035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfae2e9f-1379-46dc-9a88-be2bc9e98de8 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:47.980682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae2e9f-1379-46dc-9a88-be2bc9e98de8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:47.980986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae2e9f-1379-46dc-9a88-be2bc9e98de8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:47.982373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfae2e9f-1379-46dc-9a88-be2bc9e98de8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:47.994784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 251/500] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:47.995679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae2e9f-1379-46dc-9a88-be2bc9e98de8 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.079140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a29230-5709-44e6-8898-de38299ea595 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.092288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfae2e9f-1379-46dc-9a88-be2bc9e98de8 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.106800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:48.114024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adb7edd-3bc2-42ea-9cc1-b20d83065583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.114746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb7edd-3bc2-42ea-9cc1-b20d83065583 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.115021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb7edd-3bc2-42ea-9cc1-b20d83065583 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.115289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adb7edd-3bc2-42ea-9cc1-b20d83065583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.125066 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb7edd-3bc2-42ea-9cc1-b20d83065583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:48.126492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adb7edd-3bc2-42ea-9cc1-b20d83065583 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:48.126594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbf04b-fdd1-4cc6-aeb4-23d56d75a18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.127199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbf04b-fdd1-4cc6-aeb4-23d56d75a18d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.127502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbf04b-fdd1-4cc6-aeb4-23d56d75a18d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.127778 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbf04b-fdd1-4cc6-aeb4-23d56d75a18d None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.128438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbf04b-fdd1-4cc6-aeb4-23d56d75a18d None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:48.128730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbf04b-fdd1-4cc6-aeb4-23d56d75a18d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:48.130105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbf04b-fdd1-4cc6-aeb4-23d56d75a18d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:48.137843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 252/501] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:48.138619 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbf04b-fdd1-4cc6-aeb4-23d56d75a18d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-0adb7edd-3bc2-42ea-9cc1-b20d83065583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:48.141328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:48.142544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:48.143598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:48.143598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:48.143598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:48.143598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 250/502] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:48.143947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759866e6-a454-4001-b78c-91ec4cf766f0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.144184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759866e6-a454-4001-b78c-91ec4cf766f0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.144463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759866e6-a454-4001-b78c-91ec4cf766f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.162784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-759866e6-a454-4001-b78c-91ec4cf766f0 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:48.163995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-759866e6-a454-4001-b78c-91ec4cf766f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:48.184183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:48.230818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 251/503] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:48.231606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759866e6-a454-4001-b78c-91ec4cf766f0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.243165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbf04b-fdd1-4cc6-aeb4-23d56d75a18d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.277854 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ab5ea-947e-4115-a5a8-5bd3c5661187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.278479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ab5ea-947e-4115-a5a8-5bd3c5661187 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.278760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ab5ea-947e-4115-a5a8-5bd3c5661187 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.279201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841ab5ea-947e-4115-a5a8-5bd3c5661187 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.289907 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ab5ea-947e-4115-a5a8-5bd3c5661187 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:48.291827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841ab5ea-947e-4115-a5a8-5bd3c5661187 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-841ab5ea-947e-4115-a5a8-5bd3c5661187 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:00:48.305475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:48.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:48.307248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 253/504] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 10 14:00:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:48.307902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ab5ea-947e-4115-a5a8-5bd3c5661187 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.346285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:48.354662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759866e6-a454-4001-b78c-91ec4cf766f0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.389972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1b113-f1cb-493e-b000-8b18f3f97d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.390579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1b113-f1cb-493e-b000-8b18f3f97d26 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.390854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1b113-f1cb-493e-b000-8b18f3f97d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.391128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a1b113-f1cb-493e-b000-8b18f3f97d26 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.398940 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1b113-f1cb-493e-b000-8b18f3f97d26 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:48.400407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a1b113-f1cb-493e-b000-8b18f3f97d26 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-20a1b113-f1cb-493e-b000-8b18f3f97d26 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:00:48.413930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:48.415345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 252/505] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:48 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:48.416668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1b113-f1cb-493e-b000-8b18f3f97d26 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.436066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841ab5ea-947e-4115-a5a8-5bd3c5661187 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.440198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:48.474785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f23e8-c26c-41a0-9183-c16a1f4b78a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.475414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f23e8-c26c-41a0-9183-c16a1f4b78a8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.475715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f23e8-c26c-41a0-9183-c16a1f4b78a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.475980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f23e8-c26c-41a0-9183-c16a1f4b78a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.487053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f23e8-c26c-41a0-9183-c16a1f4b78a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:48.488302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20f23e8-c26c-41a0-9183-c16a1f4b78a8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-b20f23e8-c26c-41a0-9183-c16a1f4b78a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:48.503832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:48.505478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:48.506654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:48.506654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:48.506654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:48.506654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 254/506] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:48.506654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f23e8-c26c-41a0-9183-c16a1f4b78a8 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.528901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a1b113-f1cb-493e-b000-8b18f3f97d26 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.563731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a0b66-4f55-4812-9e4e-94f353841755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.564329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a0b66-4f55-4812-9e4e-94f353841755 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.564607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a0b66-4f55-4812-9e4e-94f353841755 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.564878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a0b66-4f55-4812-9e4e-94f353841755 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.565507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a0b66-4f55-4812-9e4e-94f353841755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:48.565803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a0b66-4f55-4812-9e4e-94f353841755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:48.567770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a0b66-4f55-4812-9e4e-94f353841755 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:48.579173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 253/507] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:48.579978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a0b66-4f55-4812-9e4e-94f353841755 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.602156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:48.614698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20f23e8-c26c-41a0-9183-c16a1f4b78a8 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.646809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d3c2a1-1eec-4efe-a0a0-ee3799187fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.647403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3c2a1-1eec-4efe-a0a0-ee3799187fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.647714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3c2a1-1eec-4efe-a0a0-ee3799187fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.647981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d3c2a1-1eec-4efe-a0a0-ee3799187fc2 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.648692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d3c2a1-1eec-4efe-a0a0-ee3799187fc2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:48.649031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d3c2a1-1eec-4efe-a0a0-ee3799187fc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:48.650421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d3c2a1-1eec-4efe-a0a0-ee3799187fc2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:48.656264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:48.680632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a0b66-4f55-4812-9e4e-94f353841755 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.686916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 255/508] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 10 14:00:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:48.687770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d3c2a1-1eec-4efe-a0a0-ee3799187fc2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.714727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f653bc-7888-4fbe-b122-f15bb9bb7200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.715323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f653bc-7888-4fbe-b122-f15bb9bb7200 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.715623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f653bc-7888-4fbe-b122-f15bb9bb7200 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.715889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f653bc-7888-4fbe-b122-f15bb9bb7200 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.716600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f653bc-7888-4fbe-b122-f15bb9bb7200 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:48.716900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f653bc-7888-4fbe-b122-f15bb9bb7200 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:48.718317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f653bc-7888-4fbe-b122-f15bb9bb7200 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:48.754193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 254/509] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 10 14:00:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:48.755771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f653bc-7888-4fbe-b122-f15bb9bb7200 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.786945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d3c2a1-1eec-4efe-a0a0-ee3799187fc2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.820697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391ea01-382e-45af-8b85-de4bcad7bcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.821320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391ea01-382e-45af-8b85-de4bcad7bcfd None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.821661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391ea01-382e-45af-8b85-de4bcad7bcfd None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.821941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4391ea01-382e-45af-8b85-de4bcad7bcfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.832004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391ea01-382e-45af-8b85-de4bcad7bcfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:48.833596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4391ea01-382e-45af-8b85-de4bcad7bcfd None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:48.840325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-4391ea01-382e-45af-8b85-de4bcad7bcfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:48.849195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:48.850284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:48.851116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:48.851116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:48.851116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:48.851116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 256/510] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:48.851116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391ea01-382e-45af-8b85-de4bcad7bcfd None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.863062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f653bc-7888-4fbe-b122-f15bb9bb7200 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.898133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9c8b3-b1e0-4d45-907c-4267d0971670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.898732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9c8b3-b1e0-4d45-907c-4267d0971670 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.899019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9c8b3-b1e0-4d45-907c-4267d0971670 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.899289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df9c8b3-b1e0-4d45-907c-4267d0971670 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.899984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9c8b3-b1e0-4d45-907c-4267d0971670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:48.900292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9c8b3-b1e0-4d45-907c-4267d0971670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:48.901676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df9c8b3-b1e0-4d45-907c-4267d0971670 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:48.903073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:48.914630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 255/511] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:48.915428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9c8b3-b1e0-4d45-907c-4267d0971670 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:48.956250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4391ea01-382e-45af-8b85-de4bcad7bcfd None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:48.990566 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb840eb-10f1-4cdd-a999-fdff4df6b5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:48.991210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb840eb-10f1-4cdd-a999-fdff4df6b5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:48.991509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb840eb-10f1-4cdd-a999-fdff4df6b5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:48.991782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb840eb-10f1-4cdd-a999-fdff4df6b5e0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:48.998665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb840eb-10f1-4cdd-a999-fdff4df6b5e0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:49.000185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb840eb-10f1-4cdd-a999-fdff4df6b5e0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-0eb840eb-10f1-4cdd-a999-fdff4df6b5e0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:00:49.013264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:49.014201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 257/512] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:49.015099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb840eb-10f1-4cdd-a999-fdff4df6b5e0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:49.022552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df9c8b3-b1e0-4d45-907c-4267d0971670 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:49.055276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6e691-f9bb-4542-95d4-44abf4500450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:49.055891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6e691-f9bb-4542-95d4-44abf4500450 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:49.056162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6e691-f9bb-4542-95d4-44abf4500450 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:49.056425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd6e691-f9bb-4542-95d4-44abf4500450 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:49.071515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:49.116103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb840eb-10f1-4cdd-a999-fdff4df6b5e0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:49.118032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:49.126044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6e691-f9bb-4542-95d4-44abf4500450 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:49.127576 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd6e691-f9bb-4542-95d4-44abf4500450 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:49.149821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a0342-7ea5-4976-912b-0096db922fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:49.150422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a0342-7ea5-4976-912b-0096db922fab None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:49.150701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a0342-7ea5-4976-912b-0096db922fab None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:49.150974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a0342-7ea5-4976-912b-0096db922fab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:49.217583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a0342-7ea5-4976-912b-0096db922fab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:49.219044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a0342-7ea5-4976-912b-0096db922fab None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:49.224579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 256/513] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:49.225400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6e691-f9bb-4542-95d4-44abf4500450 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:49.277063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:49.315962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:49.320127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 258/514] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:49.320955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a0342-7ea5-4976-912b-0096db922fab None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:49.324219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd6e691-f9bb-4542-95d4-44abf4500450 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:49.358843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b417ab5-717e-4d2e-9590-6fbd9d137477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:49.359433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b417ab5-717e-4d2e-9590-6fbd9d137477 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:49.359730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b417ab5-717e-4d2e-9590-6fbd9d137477 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:49.359995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b417ab5-717e-4d2e-9590-6fbd9d137477 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:49.360629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b417ab5-717e-4d2e-9590-6fbd9d137477 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:49.360924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b417ab5-717e-4d2e-9590-6fbd9d137477 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:49.362287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b417ab5-717e-4d2e-9590-6fbd9d137477 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:49.373391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 257/515] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:49.374162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b417ab5-717e-4d2e-9590-6fbd9d137477 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:49.422092 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a0342-7ea5-4976-912b-0096db922fab None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:49.457995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320afb9-a9df-4c8e-a152-e7eddf6061c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:49.458615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320afb9-a9df-4c8e-a152-e7eddf6061c3 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:49.458890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320afb9-a9df-4c8e-a152-e7eddf6061c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:49.459152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2320afb9-a9df-4c8e-a152-e7eddf6061c3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:49.465980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320afb9-a9df-4c8e-a152-e7eddf6061c3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:49.467404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2320afb9-a9df-4c8e-a152-e7eddf6061c3 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:49.477999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b417ab5-717e-4d2e-9590-6fbd9d137477 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-2320afb9-a9df-4c8e-a152-e7eddf6061c3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:00:49.478755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:49.479826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 259/516] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:49.480701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320afb9-a9df-4c8e-a152-e7eddf6061c3 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:49.499231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:49.515359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9f3f6-7ffd-4d7e-8292-f610d1d9b7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:49.515993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f3f6-7ffd-4d7e-8292-f610d1d9b7c6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:49.516270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f3f6-7ffd-4d7e-8292-f610d1d9b7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:49.516540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de9f3f6-7ffd-4d7e-8292-f610d1d9b7c6 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:49.517266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9f3f6-7ffd-4d7e-8292-f610d1d9b7c6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:49.517571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9f3f6-7ffd-4d7e-8292-f610d1d9b7c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:49.519011 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de9f3f6-7ffd-4d7e-8292-f610d1d9b7c6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:49.538582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 258/517] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 10 14:00:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:49.539309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9f3f6-7ffd-4d7e-8292-f610d1d9b7c6 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:49.553894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:49.584069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2320afb9-a9df-4c8e-a152-e7eddf6061c3 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:49.618997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bad8fb-fbf1-4426-9fe1-e4664a2f73f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:49.619659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bad8fb-fbf1-4426-9fe1-e4664a2f73f6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:49.619938 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bad8fb-fbf1-4426-9fe1-e4664a2f73f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:49.620212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bad8fb-fbf1-4426-9fe1-e4664a2f73f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:49.631110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bad8fb-fbf1-4426-9fe1-e4664a2f73f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:49.632608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bad8fb-fbf1-4426-9fe1-e4664a2f73f6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:49.638598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de9f3f6-7ffd-4d7e-8292-f610d1d9b7c6 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-e9bad8fb-fbf1-4426-9fe1-e4664a2f73f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:49.646435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:49.647593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:49.648625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:49.648625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:49.648625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:49.648625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 260/518] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:49.648625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bad8fb-fbf1-4426-9fe1-e4664a2f73f6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:49.673283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226f368-8026-489d-b87e-63b9d847a5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:49.673904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226f368-8026-489d-b87e-63b9d847a5d8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:49.674175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226f368-8026-489d-b87e-63b9d847a5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:49.674443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226f368-8026-489d-b87e-63b9d847a5d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:49.684820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226f368-8026-489d-b87e-63b9d847a5d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:49.686262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226f368-8026-489d-b87e-63b9d847a5d8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-4226f368-8026-489d-b87e-63b9d847a5d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:49.701298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:49.702372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:49.703223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:49.703223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:49.703223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:49.703223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 259/519] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:49.703563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226f368-8026-489d-b87e-63b9d847a5d8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:49.720171 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:49.751428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bad8fb-fbf1-4426-9fe1-e4664a2f73f6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:49.767434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:49.784788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50763b5-33f9-4f50-9a89-8880659f6901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:49.785390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50763b5-33f9-4f50-9a89-8880659f6901 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:49.785685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50763b5-33f9-4f50-9a89-8880659f6901 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:49.785949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50763b5-33f9-4f50-9a89-8880659f6901 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:49.786585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50763b5-33f9-4f50-9a89-8880659f6901 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:49.788085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50763b5-33f9-4f50-9a89-8880659f6901 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:49.807986 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226f368-8026-489d-b87e-63b9d847a5d8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:49.809385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 261/520] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:00:49 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:00:49.810187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50763b5-33f9-4f50-9a89-8880659f6901 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:49.848434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9cd38-f652-4fcb-85ed-ad7b20885666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:49.849061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9cd38-f652-4fcb-85ed-ad7b20885666 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:49.849366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9cd38-f652-4fcb-85ed-ad7b20885666 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:49.849643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb9cd38-f652-4fcb-85ed-ad7b20885666 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:49.850450 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9cd38-f652-4fcb-85ed-ad7b20885666 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:49.850757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9cd38-f652-4fcb-85ed-ad7b20885666 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:49.852278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb9cd38-f652-4fcb-85ed-ad7b20885666 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:49.876374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 260/521] 10.222.0.26 () {56 vars in 1139 bytes} [Sat Sep 10 14:00:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:49.876374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9cd38-f652-4fcb-85ed-ad7b20885666 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:49.934797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50763b5-33f9-4f50-9a89-8880659f6901 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:49.965580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:49.977708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b35a0-4e0e-4fdb-a77a-292d50504cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:49.978324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b35a0-4e0e-4fdb-a77a-292d50504cab None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:49.978600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b35a0-4e0e-4fdb-a77a-292d50504cab None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:49.978874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4b35a0-4e0e-4fdb-a77a-292d50504cab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:49.984212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb9cd38-f652-4fcb-85ed-ad7b20885666 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:50.010740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:50.027979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3317cf1-7142-43eb-9acd-dd688d853612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:50.028843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3317cf1-7142-43eb-9acd-dd688d853612 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:50.029271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3317cf1-7142-43eb-9acd-dd688d853612 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:50.029691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3317cf1-7142-43eb-9acd-dd688d853612 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:50.030504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3317cf1-7142-43eb-9acd-dd688d853612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:50.030809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3317cf1-7142-43eb-9acd-dd688d853612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:50.032283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3317cf1-7142-43eb-9acd-dd688d853612 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:50.044143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 261/522] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:50.045054 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3317cf1-7142-43eb-9acd-dd688d853612 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:50.063651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b35a0-4e0e-4fdb-a77a-292d50504cab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:50.065057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4b35a0-4e0e-4fdb-a77a-292d50504cab None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:50.148604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3317cf1-7142-43eb-9acd-dd688d853612 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:50.177800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 262/523] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:50.178775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b35a0-4e0e-4fdb-a77a-292d50504cab None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:50.184622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e5d92-adf5-4adb-bc19-3ae1caec6e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:50.185420 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e5d92-adf5-4adb-bc19-3ae1caec6e50 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:50.185767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e5d92-adf5-4adb-bc19-3ae1caec6e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:50.186105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e5d92-adf5-4adb-bc19-3ae1caec6e50 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:50.186823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e5d92-adf5-4adb-bc19-3ae1caec6e50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:50.187193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e5d92-adf5-4adb-bc19-3ae1caec6e50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:50.188677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22e5d92-adf5-4adb-bc19-3ae1caec6e50 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:50.205854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 262/524] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:50.205854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e5d92-adf5-4adb-bc19-3ae1caec6e50 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:50.212812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:50.251786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:50.294869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4b35a0-4e0e-4fdb-a77a-292d50504cab None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:50.309184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22e5d92-adf5-4adb-bc19-3ae1caec6e50 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:50.331245 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab26a6-b2a3-42ed-91e1-fb456a31dd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:50.331810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab26a6-b2a3-42ed-91e1-fb456a31dd6e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:50.332089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab26a6-b2a3-42ed-91e1-fb456a31dd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:50.332364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ab26a6-b2a3-42ed-91e1-fb456a31dd6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:50.345526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68a43b-8857-4253-b86f-209e4832d399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:50.346419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68a43b-8857-4253-b86f-209e4832d399 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:50.346690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68a43b-8857-4253-b86f-209e4832d399 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:50.346955 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce68a43b-8857-4253-b86f-209e4832d399 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:50.354026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68a43b-8857-4253-b86f-209e4832d399 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:50.355493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce68a43b-8857-4253-b86f-209e4832d399 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-ce68a43b-8857-4253-b86f-209e4832d399 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:00:50.367982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:50.369006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:50.370121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:00:50.370121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:00:50.370121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:00:50.370121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:00:50.370121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:00:50.370121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:00:50.370121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:50.370121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:50.373821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 263/525] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:50.374561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68a43b-8857-4253-b86f-209e4832d399 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:50.409463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab26a6-b2a3-42ed-91e1-fb456a31dd6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:50.411169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ab26a6-b2a3-42ed-91e1-fb456a31dd6e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:50.441274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:50.486678 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce68a43b-8857-4253-b86f-209e4832d399 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:50.489702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:50.522780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 263/526] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:50.523654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab26a6-b2a3-42ed-91e1-fb456a31dd6e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:50.528120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faca4c5-4757-43cf-a4bc-607e56de2b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:50.529014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faca4c5-4757-43cf-a4bc-607e56de2b97 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:50.529483 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faca4c5-4757-43cf-a4bc-607e56de2b97 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:50.529962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faca4c5-4757-43cf-a4bc-607e56de2b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:50.546126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faca4c5-4757-43cf-a4bc-607e56de2b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:50.548741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faca4c5-4757-43cf-a4bc-607e56de2b97 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-0faca4c5-4757-43cf-a4bc-607e56de2b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:50.568412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:50.569395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:50.570303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:50.570303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:50.570303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:50.574058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 264/527] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:50.574870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faca4c5-4757-43cf-a4bc-607e56de2b97 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:50.635394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ab26a6-b2a3-42ed-91e1-fb456a31dd6e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:50.667046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:50.677716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d253d-ddd6-4ffe-bd50-30093f67b76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:50.678340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d253d-ddd6-4ffe-bd50-30093f67b76f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:50.678615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d253d-ddd6-4ffe-bd50-30093f67b76f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:50.678887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7d253d-ddd6-4ffe-bd50-30093f67b76f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:50.684238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faca4c5-4757-43cf-a4bc-607e56de2b97 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:50.685668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d253d-ddd6-4ffe-bd50-30093f67b76f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:50.687296 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7d253d-ddd6-4ffe-bd50-30093f67b76f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-5a7d253d-ddd6-4ffe-bd50-30093f67b76f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:00:50.699473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:50.700309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:50.701147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:00:50.701147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:00:50.701147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:00:50.701147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:00:50.701147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:00:50.701147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:00:50.701147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:50.701147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:50.701147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 264/528] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:50.701608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d253d-ddd6-4ffe-bd50-30093f67b76f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:50.726005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed694ba8-0d95-43fb-85ce-d8a895f16e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:50.726960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed694ba8-0d95-43fb-85ce-d8a895f16e1e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:50.727471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed694ba8-0d95-43fb-85ce-d8a895f16e1e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:50.727947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed694ba8-0d95-43fb-85ce-d8a895f16e1e None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:50.728846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed694ba8-0d95-43fb-85ce-d8a895f16e1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:50.729157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed694ba8-0d95-43fb-85ce-d8a895f16e1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:50.730645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed694ba8-0d95-43fb-85ce-d8a895f16e1e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:50.756108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 265/529] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:50.756108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed694ba8-0d95-43fb-85ce-d8a895f16e1e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:50.773651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:50.819715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7d253d-ddd6-4ffe-bd50-30093f67b76f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:50.855557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ba2ff-b5bf-4c17-9207-6ffa926932cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:50.856294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ba2ff-b5bf-4c17-9207-6ffa926932cb None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:50.856571 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ba2ff-b5bf-4c17-9207-6ffa926932cb None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:50.856843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ba2ff-b5bf-4c17-9207-6ffa926932cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:50.862109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed694ba8-0d95-43fb-85ce-d8a895f16e1e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:50.868264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ba2ff-b5bf-4c17-9207-6ffa926932cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:50.869705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4ba2ff-b5bf-4c17-9207-6ffa926932cb None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-dc4ba2ff-b5bf-4c17-9207-6ffa926932cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:50.885614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:50.886491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:50.887315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:50.887315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:50.887315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:50.887315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 265/530] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:50.887977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ba2ff-b5bf-4c17-9207-6ffa926932cb None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:50.901811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149df45-4b29-4f4e-9788-9ef1138d0856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:50.902413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149df45-4b29-4f4e-9788-9ef1138d0856 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:50.902686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149df45-4b29-4f4e-9788-9ef1138d0856 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:50.902953 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9149df45-4b29-4f4e-9788-9ef1138d0856 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:50.903629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149df45-4b29-4f4e-9788-9ef1138d0856 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:50.903918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149df45-4b29-4f4e-9788-9ef1138d0856 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:50.905279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9149df45-4b29-4f4e-9788-9ef1138d0856 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:50.916601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 266/531] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:50.917380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149df45-4b29-4f4e-9788-9ef1138d0856 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:50.934611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:50.998746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4ba2ff-b5bf-4c17-9207-6ffa926932cb None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:50.999527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:51.026409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9149df45-4b29-4f4e-9788-9ef1138d0856 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:51.036295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e0bef-4dd4-4746-82c7-d908b7ed8cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:51.036946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e0bef-4dd4-4746-82c7-d908b7ed8cbb None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:51.037232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e0bef-4dd4-4746-82c7-d908b7ed8cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:51.037503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921e0bef-4dd4-4746-82c7-d908b7ed8cbb None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:51.038241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e0bef-4dd4-4746-82c7-d908b7ed8cbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:51.038550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e0bef-4dd4-4746-82c7-d908b7ed8cbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:51.040202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921e0bef-4dd4-4746-82c7-d908b7ed8cbb None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:51.059771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 266/532] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:00:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:51.060927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e0bef-4dd4-4746-82c7-d908b7ed8cbb None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:51.066257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95573ad-0d14-4c2d-9d5f-be30cf03ba38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:51.067530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95573ad-0d14-4c2d-9d5f-be30cf03ba38 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:51.067881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95573ad-0d14-4c2d-9d5f-be30cf03ba38 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:51.068371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95573ad-0d14-4c2d-9d5f-be30cf03ba38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:51.069173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95573ad-0d14-4c2d-9d5f-be30cf03ba38 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=cb24a1b55ee7408ab24c68991aec54ec) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:51.069571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95573ad-0d14-4c2d-9d5f-be30cf03ba38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:51.071110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95573ad-0d14-4c2d-9d5f-be30cf03ba38 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:51.077178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 267/533] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:50 2022] GET /identity/v3/role_assignments?user.id=cb24a1b55ee7408ab24c68991aec54ec&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:51.084552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95573ad-0d14-4c2d-9d5f-be30cf03ba38 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:51.193142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:51.198765 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921e0bef-4dd4-4746-82c7-d908b7ed8cbb None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:51.246884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1133278-4539-4c3f-8921-758132135b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:51.247542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1133278-4539-4c3f-8921-758132135b8c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:51.247826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1133278-4539-4c3f-8921-758132135b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:51.248154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1133278-4539-4c3f-8921-758132135b8c None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:51.248754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1133278-4539-4c3f-8921-758132135b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:51.249057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1133278-4539-4c3f-8921-758132135b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:51.250784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1133278-4539-4c3f-8921-758132135b8c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:51.258035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95573ad-0d14-4c2d-9d5f-be30cf03ba38 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:51.267072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 267/534] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:51.268502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1133278-4539-4c3f-8921-758132135b8c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:51.305231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9452a-4e74-44f8-bb9d-70e26f6279df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:51.306824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9452a-4e74-44f8-bb9d-70e26f6279df None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:51.307611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9452a-4e74-44f8-bb9d-70e26f6279df None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:51.308092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b9452a-4e74-44f8-bb9d-70e26f6279df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:51.318360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9452a-4e74-44f8-bb9d-70e26f6279df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:51.320925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b9452a-4e74-44f8-bb9d-70e26f6279df None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-f7b9452a-4e74-44f8-bb9d-70e26f6279df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:00:51.337779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:51.338647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:51.339528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:00:51.339528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:00:51.339528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:00:51.339528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:00:51.339528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:00:51.339528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:00:51.339528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:51.339528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:51.345724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 268/535] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:51.345724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:51.346452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9452a-4e74-44f8-bb9d-70e26f6279df None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:51.386788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1133278-4539-4c3f-8921-758132135b8c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:51.423727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59ffd6-c0c8-4bf5-98c2-c9e0cc738e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:51.424335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59ffd6-c0c8-4bf5-98c2-c9e0cc738e27 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:51.424609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59ffd6-c0c8-4bf5-98c2-c9e0cc738e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:51.424873 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59ffd6-c0c8-4bf5-98c2-c9e0cc738e27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:51.425753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59ffd6-c0c8-4bf5-98c2-c9e0cc738e27 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=cf75b583add44b6b87bff5dc67080d1c) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:51.425985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59ffd6-c0c8-4bf5-98c2-c9e0cc738e27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:51.428244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59ffd6-c0c8-4bf5-98c2-c9e0cc738e27 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:51.443934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 268/536] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:51 2022] GET /identity/v3/role_assignments?user.id=cf75b583add44b6b87bff5dc67080d1c&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:51.445629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59ffd6-c0c8-4bf5-98c2-c9e0cc738e27 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:51.485860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b9452a-4e74-44f8-bb9d-70e26f6279df None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:51.500939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:51.533367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58935c86-6fc8-45dc-bbeb-b1a4335e5d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:51.534709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58935c86-6fc8-45dc-bbeb-b1a4335e5d01 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:51.535193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58935c86-6fc8-45dc-bbeb-b1a4335e5d01 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:51.535843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58935c86-6fc8-45dc-bbeb-b1a4335e5d01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:51.545181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58935c86-6fc8-45dc-bbeb-b1a4335e5d01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:51.547364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58935c86-6fc8-45dc-bbeb-b1a4335e5d01 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-58935c86-6fc8-45dc-bbeb-b1a4335e5d01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:00:51.562525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:51.563476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:51.564316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:00:51.564316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:00:51.564316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:00:51.564316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:00:51.564316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:00:51.564316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:00:51.564316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:51.564316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:51.568065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 269/537] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:51.573633 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58935c86-6fc8-45dc-bbeb-b1a4335e5d01 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:51.677211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:51.703686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58935c86-6fc8-45dc-bbeb-b1a4335e5d01 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:51.710122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59ffd6-c0c8-4bf5-98c2-c9e0cc738e27 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:51.741052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e358bd-afd3-48a1-a281-e5007bc5b030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:51.742234 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e358bd-afd3-48a1-a281-e5007bc5b030 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:51.742616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e358bd-afd3-48a1-a281-e5007bc5b030 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:51.742992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e358bd-afd3-48a1-a281-e5007bc5b030 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:51.743825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e358bd-afd3-48a1-a281-e5007bc5b030 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:51.744242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e358bd-afd3-48a1-a281-e5007bc5b030 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:51.745907 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e358bd-afd3-48a1-a281-e5007bc5b030 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:51.760121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 270/538] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:00:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:51.765842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e358bd-afd3-48a1-a281-e5007bc5b030 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:51.792413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4ef5a-abda-4215-8e56-1cf1b58cb4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:51.793025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4ef5a-abda-4215-8e56-1cf1b58cb4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:51.793304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4ef5a-abda-4215-8e56-1cf1b58cb4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:51.793573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4ef5a-abda-4215-8e56-1cf1b58cb4c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:51.794273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4ef5a-abda-4215-8e56-1cf1b58cb4c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:51.794579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4ef5a-abda-4215-8e56-1cf1b58cb4c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:51.796049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4ef5a-abda-4215-8e56-1cf1b58cb4c6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:51.820755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 269/539] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:00:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:51.826920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4ef5a-abda-4215-8e56-1cf1b58cb4c6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:51.925650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:51.950957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4ef5a-abda-4215-8e56-1cf1b58cb4c6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:51.998621 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad078aa-3b30-439c-a29a-68c1ddee1d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:51.999528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad078aa-3b30-439c-a29a-68c1ddee1d92 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:52.000543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad078aa-3b30-439c-a29a-68c1ddee1d92 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:52.001140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad078aa-3b30-439c-a29a-68c1ddee1d92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:52.003460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad078aa-3b30-439c-a29a-68c1ddee1d92 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=fdc278f50e794e12a27d30fca9d131c1) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:52.003943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad078aa-3b30-439c-a29a-68c1ddee1d92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:52.005065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e358bd-afd3-48a1-a281-e5007bc5b030 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:52.007985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad078aa-3b30-439c-a29a-68c1ddee1d92 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:52.019008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 270/540] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:51 2022] GET /identity/v3/role_assignments?user.id=fdc278f50e794e12a27d30fca9d131c1&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:52.033260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:52.054136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa323c1b-9c2f-459c-8da0-e9fa06528e81 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:52.054457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa323c1b-9c2f-459c-8da0-e9fa06528e81 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:52.054683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa323c1b-9c2f-459c-8da0-e9fa06528e81 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:52.056426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 271/541] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:00:52.056902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14970ac2-0888-474c-ab94-61a1dc994ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:52.057757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14970ac2-0888-474c-ab94-61a1dc994ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:52.058043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14970ac2-0888-474c-ab94-61a1dc994ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:52.058412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14970ac2-0888-474c-ab94-61a1dc994ee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:52.059274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14970ac2-0888-474c-ab94-61a1dc994ee3 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=12c71caf31354ec193e3e2da231f1e77) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:52.059643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14970ac2-0888-474c-ab94-61a1dc994ee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:52.061419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14970ac2-0888-474c-ab94-61a1dc994ee3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:52.065882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a7b82-888d-45b3-adde-4da8c8eb6d5e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:52.066157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a7b82-888d-45b3-adde-4da8c8eb6d5e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:52.066432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a7b82-888d-45b3-adde-4da8c8eb6d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:52.079235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 271/542] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:51 2022] GET /identity/v3/role_assignments?user.id=12c71caf31354ec193e3e2da231f1e77&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:52.088340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-b82a7b82-888d-45b3-adde-4da8c8eb6d5e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:52.089616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b82a7b82-888d-45b3-adde-4da8c8eb6d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:52.174769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 272/543] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:52.515764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14970ac2-0888-474c-ab94-61a1dc994ee3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:52.656057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14970ac2-0888-474c-ab94-61a1dc994ee3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:52.702451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a6f5a-41d6-4661-ac8f-334e310cea90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:52.702994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a6f5a-41d6-4661-ac8f-334e310cea90 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:52.703266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a6f5a-41d6-4661-ac8f-334e310cea90 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:52.703554 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a6f5a-41d6-4661-ac8f-334e310cea90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:52.724366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a6f5a-41d6-4661-ac8f-334e310cea90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a53a6f5a-41d6-4661-ac8f-334e310cea90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:52.729617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:52.730473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:52.731249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 272/544] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:52.736689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a7b82-888d-45b3-adde-4da8c8eb6d5e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:52.843832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:52.875663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a7b82-888d-45b3-adde-4da8c8eb6d5e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:52.921016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9ccc0-f2df-45a0-a2c6-02c815c041b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:52.921636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9ccc0-f2df-45a0-a2c6-02c815c041b2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:52.921914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9ccc0-f2df-45a0-a2c6-02c815c041b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:52.922188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9ccc0-f2df-45a0-a2c6-02c815c041b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:52.922869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9ccc0-f2df-45a0-a2c6-02c815c041b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:52.923235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9ccc0-f2df-45a0-a2c6-02c815c041b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:52.924647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9ccc0-f2df-45a0-a2c6-02c815c041b2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:52.933259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 273/545] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:00:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:52.938996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a6f5a-41d6-4661-ac8f-334e310cea90 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:53.025133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:53.097181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a6f5a-41d6-4661-ac8f-334e310cea90 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:53.148173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d7d05-efae-4265-9506-033cd32895de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:53.148808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d7d05-efae-4265-9506-033cd32895de None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:53.149087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d7d05-efae-4265-9506-033cd32895de None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:53.149357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3d7d05-efae-4265-9506-033cd32895de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:53.167158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d7d05-efae-4265-9506-033cd32895de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:53.168651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3d7d05-efae-4265-9506-033cd32895de None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9d3d7d05-efae-4265-9506-033cd32895de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:53.188393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:53.189346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:53.190171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:53.190171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:53.190171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:53.190171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 273/546] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:53.195803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9ccc0-f2df-45a0-a2c6-02c815c041b2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:53.329480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9ccc0-f2df-45a0-a2c6-02c815c041b2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:53.367500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd7002-9c51-48ee-ac93-49b2f19851f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:53.368117 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd7002-9c51-48ee-ac93-49b2f19851f6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:53.368390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd7002-9c51-48ee-ac93-49b2f19851f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:53.368658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dd7002-9c51-48ee-ac93-49b2f19851f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:53.369300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd7002-9c51-48ee-ac93-49b2f19851f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:53.369600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd7002-9c51-48ee-ac93-49b2f19851f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:53.370961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dd7002-9c51-48ee-ac93-49b2f19851f6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:53.381500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:53.390783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 274/547] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:53.396922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d7d05-efae-4265-9506-033cd32895de None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:53.546829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:53.553623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3d7d05-efae-4265-9506-033cd32895de None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:53.596730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3b8db-6e44-499f-8623-89d7da08afcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:53.597272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3b8db-6e44-499f-8623-89d7da08afcd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:53.597549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3b8db-6e44-499f-8623-89d7da08afcd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:53.597893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3b8db-6e44-499f-8623-89d7da08afcd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:53.606476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3b8db-6e44-499f-8623-89d7da08afcd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:53.607968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3b8db-6e44-499f-8623-89d7da08afcd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c5b3b8db-6e44-499f-8623-89d7da08afcd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:00:53.619890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:53.621296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:53.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 274/548] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:00:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:53.626906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd7002-9c51-48ee-ac93-49b2f19851f6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:53.746876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dd7002-9c51-48ee-ac93-49b2f19851f6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:53.786892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56afe47d-4763-4af7-8eef-c4335aabc206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:53.787516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afe47d-4763-4af7-8eef-c4335aabc206 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:53.787791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afe47d-4763-4af7-8eef-c4335aabc206 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:53.788059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56afe47d-4763-4af7-8eef-c4335aabc206 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:53.788779 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afe47d-4763-4af7-8eef-c4335aabc206 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:53.789076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afe47d-4763-4af7-8eef-c4335aabc206 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:53.790499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56afe47d-4763-4af7-8eef-c4335aabc206 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:53.814809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 275/549] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 10 14:00:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:53.821080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3b8db-6e44-499f-8623-89d7da08afcd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:53.962614 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3b8db-6e44-499f-8623-89d7da08afcd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:53.998105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd7d5d-ee58-4f5d-bf64-aa08bad6158c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:53.998714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd7d5d-ee58-4f5d-bf64-aa08bad6158c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:53.998995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd7d5d-ee58-4f5d-bf64-aa08bad6158c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:53.999267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd7d5d-ee58-4f5d-bf64-aa08bad6158c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.024429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:54.040396 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e3bbf-e492-4f71-a9c1-0c0ae0cc4e26 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:54.040702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e3bbf-e492-4f71-a9c1-0c0ae0cc4e26 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:54.040954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e3bbf-e492-4f71-a9c1-0c0ae0cc4e26 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.042568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 276/550] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:54.048053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5c9b6-c837-47aa-85e7-c5451d844093 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:54.048340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5c9b6-c837-47aa-85e7-c5451d844093 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:54.048615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b5c9b6-c837-47aa-85e7-c5451d844093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.074044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd7d5d-ee58-4f5d-bf64-aa08bad6158c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:54.074751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-16b5c9b6-c837-47aa-85e7-c5451d844093 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:54.075957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-16b5c9b6-c837-47aa-85e7-c5451d844093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:54.076385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd7d5d-ee58-4f5d-bf64-aa08bad6158c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:54.144750 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:54.168426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 277/551] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:54.170742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d479c-c418-4f5e-b50b-672e3d983f45 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:54.171214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d479c-c418-4f5e-b50b-672e3d983f45 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:54.171956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85d479c-c418-4f5e-b50b-672e3d983f45 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.174446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 278/552] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:54.185133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376a7c4-8504-46f6-b837-1726f097d163 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:54.185669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376a7c4-8504-46f6-b837-1726f097d163 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:54.186125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3376a7c4-8504-46f6-b837-1726f097d163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.211691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 275/553] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:00:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:00:54.217465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd7d5d-ee58-4f5d-bf64-aa08bad6158c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:54.224605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-3376a7c4-8504-46f6-b837-1726f097d163 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:54.225796 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3376a7c4-8504-46f6-b837-1726f097d163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:54.334573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:54.351095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 279/554] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:00:54.411872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd7d5d-ee58-4f5d-bf64-aa08bad6158c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:54.458786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780be1d-a4d8-47b5-8cfe-648c52462cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:54.459879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780be1d-a4d8-47b5-8cfe-648c52462cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:54.460365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780be1d-a4d8-47b5-8cfe-648c52462cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:54.460839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d780be1d-a4d8-47b5-8cfe-648c52462cc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.471694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780be1d-a4d8-47b5-8cfe-648c52462cc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:54.474032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d780be1d-a4d8-47b5-8cfe-648c52462cc3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d780be1d-a4d8-47b5-8cfe-648c52462cc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:54.489659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:54.490608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:54.491468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:54.491468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:54.491468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:54.497297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 276/555] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:00:54.499675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376a7c4-8504-46f6-b837-1726f097d163 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:54.503296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780be1d-a4d8-47b5-8cfe-648c52462cc3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:54.675501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d780be1d-a4d8-47b5-8cfe-648c52462cc3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:54.683716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:54.685466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3376a7c4-8504-46f6-b837-1726f097d163 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:54.703491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:54.719818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e5f6-b59c-490c-aaa7-389fc35ee843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:54.720572 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e5f6-b59c-490c-aaa7-389fc35ee843 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:54.720851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e5f6-b59c-490c-aaa7-389fc35ee843 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:54.721123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d3e5f6-b59c-490c-aaa7-389fc35ee843 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.724205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3464a6-39a9-4a57-9949-da14447242e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:54.724828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3464a6-39a9-4a57-9949-da14447242e7 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:54.725177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3464a6-39a9-4a57-9949-da14447242e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:54.725522 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3464a6-39a9-4a57-9949-da14447242e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.726241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3464a6-39a9-4a57-9949-da14447242e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:54.726548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3464a6-39a9-4a57-9949-da14447242e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:54.727970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3464a6-39a9-4a57-9949-da14447242e7 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:54.733010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d3e5f6-b59c-490c-aaa7-389fc35ee843 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-21d3e5f6-b59c-490c-aaa7-389fc35ee843 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:54.740700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:54.741759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:54.742914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:54.743815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 280/556] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:54.743815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3464a6-39a9-4a57-9949-da14447242e7 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:54.743967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 277/557] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:54.749193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e5f6-b59c-490c-aaa7-389fc35ee843 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:54.856151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3464a6-39a9-4a57-9949-da14447242e7 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:54.865585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d3e5f6-b59c-490c-aaa7-389fc35ee843 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:54.893712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ecfd9a-ffef-40b0-8682-1962bfbfd6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:54.894456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecfd9a-ffef-40b0-8682-1962bfbfd6e3 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:54.894752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecfd9a-ffef-40b0-8682-1962bfbfd6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:54.895072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ecfd9a-ffef-40b0-8682-1962bfbfd6e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.899807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58633fd0-02d4-4590-acf3-cd9c96898855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:54.900404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633fd0-02d4-4590-acf3-cd9c96898855 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:54.900732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633fd0-02d4-4590-acf3-cd9c96898855 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:54.900949 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58633fd0-02d4-4590-acf3-cd9c96898855 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.901603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58633fd0-02d4-4590-acf3-cd9c96898855 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:54.902094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58633fd0-02d4-4590-acf3-cd9c96898855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:54.903257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58633fd0-02d4-4590-acf3-cd9c96898855 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:54.906729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ecfd9a-ffef-40b0-8682-1962bfbfd6e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:54.911501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 278/558] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:00:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:54.912314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58633fd0-02d4-4590-acf3-cd9c96898855 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-15ecfd9a-ffef-40b0-8682-1962bfbfd6e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:54.912897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:54.913792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:54.914587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 281/559] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:54.916870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb7078-3c93-439a-9479-43f8ca6079e9 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:54.916870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb7078-3c93-439a-9479-43f8ca6079e9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:54.916870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb7078-3c93-439a-9479-43f8ca6079e9 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:54.919596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 282/560] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:54.919596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb7078-3c93-439a-9479-43f8ca6079e9 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:54.948748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:54.960977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:55.019980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58633fd0-02d4-4590-acf3-cd9c96898855 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.026252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb7078-3c93-439a-9479-43f8ca6079e9 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.053319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0992efa-4728-4685-8e3f-04ce7d0be315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:55.053933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992efa-4728-4685-8e3f-04ce7d0be315 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.054204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992efa-4728-4685-8e3f-04ce7d0be315 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.054474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0992efa-4728-4685-8e3f-04ce7d0be315 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.060966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0992efa-4728-4685-8e3f-04ce7d0be315 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:55.061166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb85dc-5bc4-49c3-898f-547a1beef8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:55.061784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb85dc-5bc4-49c3-898f-547a1beef8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.062083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb85dc-5bc4-49c3-898f-547a1beef8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.062355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb85dc-5bc4-49c3-898f-547a1beef8b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.062447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0992efa-4728-4685-8e3f-04ce7d0be315 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:55.072479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb85dc-5bc4-49c3-898f-547a1beef8b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a0992efa-4728-4685-8e3f-04ce7d0be315 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:00:55.073657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:55.074698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:55.075764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:00:55.075764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:00:55.075764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:00:55.075764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:00:55.075764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:00:55.075764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:00:55.075764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:55.075764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:55.076510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb85dc-5bc4-49c3-898f-547a1beef8b5 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:55.076587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 279/561] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:55.076587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0992efa-4728-4685-8e3f-04ce7d0be315 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-94fb85dc-5bc4-49c3-898f-547a1beef8b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:55.091593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:55.092719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:55.094195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:55.094195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:55.094195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:55.094195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 283/562] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:55.094195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5430a60-69b7-4c4e-8703-5d1f42ce94df None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.094195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5430a60-69b7-4c4e-8703-5d1f42ce94df None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.094890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5430a60-69b7-4c4e-8703-5d1f42ce94df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.111992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d5430a60-69b7-4c4e-8703-5d1f42ce94df None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:55.113180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d5430a60-69b7-4c4e-8703-5d1f42ce94df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:55.168552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:55.178908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 284/563] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:55.180230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ac64b-cd13-4f90-9c69-55447acb2984 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.180513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ac64b-cd13-4f90-9c69-55447acb2984 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.180782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ac64b-cd13-4f90-9c69-55447acb2984 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.181496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0992efa-4728-4685-8e3f-04ce7d0be315 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.182136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 285/564] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:55.182683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ac64b-cd13-4f90-9c69-55447acb2984 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:55.205162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:55.220701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d41b4-d460-4a6c-b6d8-760cd332590d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:55.221304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d41b4-d460-4a6c-b6d8-760cd332590d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.221580 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d41b4-d460-4a6c-b6d8-760cd332590d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.221851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581d41b4-d460-4a6c-b6d8-760cd332590d None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.222550 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d41b4-d460-4a6c-b6d8-760cd332590d None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:55.222806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d41b4-d460-4a6c-b6d8-760cd332590d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:55.224440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581d41b4-d460-4a6c-b6d8-760cd332590d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:55.236824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 280/565] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:00:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:55.236824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d41b4-d460-4a6c-b6d8-760cd332590d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:55.296708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502ac64b-cd13-4f90-9c69-55447acb2984 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.331324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3382a6e7-6da4-4ecf-9333-7217f594cb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:55.331955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382a6e7-6da4-4ecf-9333-7217f594cb2d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.332234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382a6e7-6da4-4ecf-9333-7217f594cb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.332652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3382a6e7-6da4-4ecf-9333-7217f594cb2d None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.333362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382a6e7-6da4-4ecf-9333-7217f594cb2d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:55.333664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382a6e7-6da4-4ecf-9333-7217f594cb2d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:55.335083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3382a6e7-6da4-4ecf-9333-7217f594cb2d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:55.342771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581d41b4-d460-4a6c-b6d8-760cd332590d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.346343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 286/566] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:00:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:55.347514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8453a-564e-4845-8dee-514b8cfcb7b3 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.347795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8453a-564e-4845-8dee-514b8cfcb7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.348128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f8453a-564e-4845-8dee-514b8cfcb7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.366338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-48f8453a-564e-4845-8dee-514b8cfcb7b3 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:55.367548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-48f8453a-564e-4845-8dee-514b8cfcb7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:55.381241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa173ab-18e5-4ed2-9c17-af4ac2594da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:55.381853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa173ab-18e5-4ed2-9c17-af4ac2594da3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.382133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa173ab-18e5-4ed2-9c17-af4ac2594da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.382407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa173ab-18e5-4ed2-9c17-af4ac2594da3 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.383052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa173ab-18e5-4ed2-9c17-af4ac2594da3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:55.383355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa173ab-18e5-4ed2-9c17-af4ac2594da3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:55.385000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa173ab-18e5-4ed2-9c17-af4ac2594da3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:55.403149 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 281/567] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:55.403149 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa173ab-18e5-4ed2-9c17-af4ac2594da3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:55.412829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:55.446189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 287/568] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:55.447220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8453a-564e-4845-8dee-514b8cfcb7b3 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:55.460718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:55.515935 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa173ab-18e5-4ed2-9c17-af4ac2594da3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.554403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7ce12-bc73-4086-b912-13282879d2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:55.554950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7ce12-bc73-4086-b912-13282879d2af None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.555221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7ce12-bc73-4086-b912-13282879d2af None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.555512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7ce12-bc73-4086-b912-13282879d2af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.564389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f8453a-564e-4845-8dee-514b8cfcb7b3 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.565805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7ce12-bc73-4086-b912-13282879d2af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:55.567314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7ce12-bc73-4086-b912-13282879d2af None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d7b7ce12-bc73-4086-b912-13282879d2af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:55.583743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:55.584700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:55.590277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:55.590277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:55.590277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:55.590277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 282/569] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:55.590277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7ce12-bc73-4086-b912-13282879d2af None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:55.606831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfecb528-4813-49f2-9380-14d2a9d33305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:55.607469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecb528-4813-49f2-9380-14d2a9d33305 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.607736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecb528-4813-49f2-9380-14d2a9d33305 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.608005 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfecb528-4813-49f2-9380-14d2a9d33305 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.608725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfecb528-4813-49f2-9380-14d2a9d33305 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=bc0a7e1b772b4703ac2e5ec33b5ecf6b) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:55.609026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfecb528-4813-49f2-9380-14d2a9d33305 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:55.610478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfecb528-4813-49f2-9380-14d2a9d33305 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:55.627476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 288/570] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:00:55 2022] GET /identity/v3/role_assignments?user.id=bc0a7e1b772b4703ac2e5ec33b5ecf6b&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:55.627476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfecb528-4813-49f2-9380-14d2a9d33305 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:55.671656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:55.720528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7ce12-bc73-4086-b912-13282879d2af None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.729288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:55.761052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea01ac-df39-461a-983e-27a0eb48fb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:55.761708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea01ac-df39-461a-983e-27a0eb48fb75 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.761991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea01ac-df39-461a-983e-27a0eb48fb75 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.762263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea01ac-df39-461a-983e-27a0eb48fb75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.764604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfecb528-4813-49f2-9380-14d2a9d33305 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.771489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea01ac-df39-461a-983e-27a0eb48fb75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:55.773252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea01ac-df39-461a-983e-27a0eb48fb75 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-20ea01ac-df39-461a-983e-27a0eb48fb75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:00:55.788789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:55.789807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 283/571] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 10 14:00:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:55.795879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea01ac-df39-461a-983e-27a0eb48fb75 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:55.815992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a805857-f034-4ba8-94ba-9774a75aea17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:55.816359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a805857-f034-4ba8-94ba-9774a75aea17 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.816640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a805857-f034-4ba8-94ba-9774a75aea17 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.816954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a805857-f034-4ba8-94ba-9774a75aea17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.829964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a805857-f034-4ba8-94ba-9774a75aea17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-1a805857-f034-4ba8-94ba-9774a75aea17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:55.836614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:55.837590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:55.838422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 289/572] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:55.839350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a805857-f034-4ba8-94ba-9774a75aea17 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:55.910186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea01ac-df39-461a-983e-27a0eb48fb75 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.934780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:55.948186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e6860-593f-455c-9fad-67bb571a9437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:55.948725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e6860-593f-455c-9fad-67bb571a9437 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:55.949006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e6860-593f-455c-9fad-67bb571a9437 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:55.949471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1e6860-593f-455c-9fad-67bb571a9437 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:55.949914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e6860-593f-455c-9fad-67bb571a9437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:55.950216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e6860-593f-455c-9fad-67bb571a9437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:55.951638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1e6860-593f-455c-9fad-67bb571a9437 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:55.955167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a805857-f034-4ba8-94ba-9774a75aea17 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:55.968122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 284/573] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:55.968122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e6860-593f-455c-9fad-67bb571a9437 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:55.991773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:56.000087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8258b8-ccb9-4db0-ba29-b2627229fe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.001003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8258b8-ccb9-4db0-ba29-b2627229fe5e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.001294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8258b8-ccb9-4db0-ba29-b2627229fe5e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.001967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8258b8-ccb9-4db0-ba29-b2627229fe5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.016859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8258b8-ccb9-4db0-ba29-b2627229fe5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-ef8258b8-ccb9-4db0-ba29-b2627229fe5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:56.022437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:56.023345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:56.024198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:56.028891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 290/574] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:56.029296 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8258b8-ccb9-4db0-ba29-b2627229fe5e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.074382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1e6860-593f-455c-9fad-67bb571a9437 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.110403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3198f-4179-4fdd-891f-3451a2742f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.111012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3198f-4179-4fdd-891f-3451a2742f96 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.111289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3198f-4179-4fdd-891f-3451a2742f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.111591 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3198f-4179-4fdd-891f-3451a2742f96 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.112317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3198f-4179-4fdd-891f-3451a2742f96 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:56.112621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3198f-4179-4fdd-891f-3451a2742f96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.114637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d3198f-4179-4fdd-891f-3451a2742f96 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:56.134417 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 285/575] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 10 14:00:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:56.139043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3198f-4179-4fdd-891f-3451a2742f96 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.150334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8258b8-ccb9-4db0-ba29-b2627229fe5e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.180838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:56.185728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c373ee1-d236-4ac4-9d25-0cc060e14f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.186329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c373ee1-d236-4ac4-9d25-0cc060e14f81 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.186603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c373ee1-d236-4ac4-9d25-0cc060e14f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.186869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c373ee1-d236-4ac4-9d25-0cc060e14f81 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.187557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c373ee1-d236-4ac4-9d25-0cc060e14f81 None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:56.187857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c373ee1-d236-4ac4-9d25-0cc060e14f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.189236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c373ee1-d236-4ac4-9d25-0cc060e14f81 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:56.198756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 291/576] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:56.199709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c373ee1-d236-4ac4-9d25-0cc060e14f81 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.252285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d3198f-4179-4fdd-891f-3451a2742f96 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.268244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:56.288013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2df88-8e40-486f-8cf0-9432d241aa55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.288642 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2df88-8e40-486f-8cf0-9432d241aa55 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.288917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2df88-8e40-486f-8cf0-9432d241aa55 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.289188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa2df88-8e40-486f-8cf0-9432d241aa55 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.297071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2df88-8e40-486f-8cf0-9432d241aa55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.298536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa2df88-8e40-486f-8cf0-9432d241aa55 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-1aa2df88-8e40-486f-8cf0-9432d241aa55 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:00:56.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:56.313295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 286/577] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:56.314283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2df88-8e40-486f-8cf0-9432d241aa55 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.323632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c373ee1-d236-4ac4-9d25-0cc060e14f81 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.359267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e3d39-36c7-40ae-918d-d7fa8c56d107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.359825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e3d39-36c7-40ae-918d-d7fa8c56d107 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.360108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e3d39-36c7-40ae-918d-d7fa8c56d107 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.360378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44e3d39-36c7-40ae-918d-d7fa8c56d107 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.361041 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e3d39-36c7-40ae-918d-d7fa8c56d107 None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:56.361343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e3d39-36c7-40ae-918d-d7fa8c56d107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.362728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44e3d39-36c7-40ae-918d-d7fa8c56d107 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:56.371800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 292/578] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:56.373009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e3d39-36c7-40ae-918d-d7fa8c56d107 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.423344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:56.427872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa2df88-8e40-486f-8cf0-9432d241aa55 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.465015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29b74c-0a0b-4d3d-a935-c13438328175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.465763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29b74c-0a0b-4d3d-a935-c13438328175 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.466044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29b74c-0a0b-4d3d-a935-c13438328175 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.466434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a29b74c-0a0b-4d3d-a935-c13438328175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.478008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29b74c-0a0b-4d3d-a935-c13438328175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.479803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a29b74c-0a0b-4d3d-a935-c13438328175 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:56.484793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44e3d39-36c7-40ae-918d-d7fa8c56d107 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-0a29b74c-0a0b-4d3d-a935-c13438328175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:56.494938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:56.495991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:56.496868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:56.496868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:56.496868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:56.496868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 287/579] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:56.497095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29b74c-0a0b-4d3d-a935-c13438328175 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.507901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:56.531286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d51932-0d52-4dff-8c7b-097f51299d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.532104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d51932-0d52-4dff-8c7b-097f51299d91 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.532385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d51932-0d52-4dff-8c7b-097f51299d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.532656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d51932-0d52-4dff-8c7b-097f51299d91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.543431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d51932-0d52-4dff-8c7b-097f51299d91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.544901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d51932-0d52-4dff-8c7b-097f51299d91 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-c1d51932-0d52-4dff-8c7b-097f51299d91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:56.560347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:56.561291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:56.562208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:56.562208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:56.562208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:56.562208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 293/580] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:56.566489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d51932-0d52-4dff-8c7b-097f51299d91 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.610254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a29b74c-0a0b-4d3d-a935-c13438328175 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.646216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa442411-6571-4bbc-b220-2634c2f82fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.646815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa442411-6571-4bbc-b220-2634c2f82fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.647095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa442411-6571-4bbc-b220-2634c2f82fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.647367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa442411-6571-4bbc-b220-2634c2f82fd2 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.648124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa442411-6571-4bbc-b220-2634c2f82fd2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:56.648431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa442411-6571-4bbc-b220-2634c2f82fd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.649874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa442411-6571-4bbc-b220-2634c2f82fd2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:56.672939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 288/581] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:56.673795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa442411-6571-4bbc-b220-2634c2f82fd2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.679660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d51932-0d52-4dff-8c7b-097f51299d91 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.685967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:56.718292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecc979-a139-4b64-9398-8e830d97e160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.718915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecc979-a139-4b64-9398-8e830d97e160 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.719189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecc979-a139-4b64-9398-8e830d97e160 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.719495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ecc979-a139-4b64-9398-8e830d97e160 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.732473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ecc979-a139-4b64-9398-8e830d97e160 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.734355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ecc979-a139-4b64-9398-8e830d97e160 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-a6ecc979-a139-4b64-9398-8e830d97e160 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:56.751568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:56.752546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:56.753997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:56.753997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:56.753997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:56.753997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 294/582] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:56.753997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecc979-a139-4b64-9398-8e830d97e160 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.777468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:56.787270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa442411-6571-4bbc-b220-2634c2f82fd2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.821920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b889dbc-6cde-4893-9278-cd6418cfcf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.822611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b889dbc-6cde-4893-9278-cd6418cfcf1a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.822894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b889dbc-6cde-4893-9278-cd6418cfcf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.823170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b889dbc-6cde-4893-9278-cd6418cfcf1a None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.823839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b889dbc-6cde-4893-9278-cd6418cfcf1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:56.824142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b889dbc-6cde-4893-9278-cd6418cfcf1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.825521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b889dbc-6cde-4893-9278-cd6418cfcf1a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:56.842367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 289/583] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:56.842367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b889dbc-6cde-4893-9278-cd6418cfcf1a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.867840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ecc979-a139-4b64-9398-8e830d97e160 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.904257 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887eec5e-6262-4596-a68b-debf69c87a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.904863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887eec5e-6262-4596-a68b-debf69c87a71 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.905202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887eec5e-6262-4596-a68b-debf69c87a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.905412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887eec5e-6262-4596-a68b-debf69c87a71 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.906042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887eec5e-6262-4596-a68b-debf69c87a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:56.906585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887eec5e-6262-4596-a68b-debf69c87a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.907759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887eec5e-6262-4596-a68b-debf69c87a71 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:56.914412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:56.924928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 295/584] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:56.924928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887eec5e-6262-4596-a68b-debf69c87a71 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:56.947127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b889dbc-6cde-4893-9278-cd6418cfcf1a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:56.982129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9659a2-6aa5-43f2-8796-e5da9eb3e2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:56.982760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9659a2-6aa5-43f2-8796-e5da9eb3e2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:56.983038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9659a2-6aa5-43f2-8796-e5da9eb3e2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:56.983310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9659a2-6aa5-43f2-8796-e5da9eb3e2e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:56.994687 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9659a2-6aa5-43f2-8796-e5da9eb3e2e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:56.996302 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9659a2-6aa5-43f2-8796-e5da9eb3e2e8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-ea9659a2-6aa5-43f2-8796-e5da9eb3e2e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:57.010882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:57.012035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:57.013108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:57.013108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:57.013108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:57.013762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:57.018901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 290/585] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:57.019617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9659a2-6aa5-43f2-8796-e5da9eb3e2e8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.037434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887eec5e-6262-4596-a68b-debf69c87a71 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.080966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b545-2f5e-4d9e-ae26-ea2cb61cef41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.081613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b545-2f5e-4d9e-ae26-ea2cb61cef41 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.081828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b545-2f5e-4d9e-ae26-ea2cb61cef41 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.082096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9706b545-2f5e-4d9e-ae26-ea2cb61cef41 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.083687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b545-2f5e-4d9e-ae26-ea2cb61cef41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:57.084301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b545-2f5e-4d9e-ae26-ea2cb61cef41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.086015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9706b545-2f5e-4d9e-ae26-ea2cb61cef41 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.097509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 296/586] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:57.102888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b545-2f5e-4d9e-ae26-ea2cb61cef41 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.125413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9659a2-6aa5-43f2-8796-e5da9eb3e2e8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.148406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:57.161049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f721ac9-2a39-4ef9-8da6-3908e6384932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.161749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f721ac9-2a39-4ef9-8da6-3908e6384932 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.162116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f721ac9-2a39-4ef9-8da6-3908e6384932 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.162458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f721ac9-2a39-4ef9-8da6-3908e6384932 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.169480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f721ac9-2a39-4ef9-8da6-3908e6384932 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.170990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f721ac9-2a39-4ef9-8da6-3908e6384932 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9f721ac9-2a39-4ef9-8da6-3908e6384932 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:00:57.183736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:57.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:57.185968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:00:57.185968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:00:57.185968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:00:57.185968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:00:57.185968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:00:57.185968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:00:57.185968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:57.185968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:57.186751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 291/587] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:57.187732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f721ac9-2a39-4ef9-8da6-3908e6384932 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.215501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9706b545-2f5e-4d9e-ae26-ea2cb61cef41 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.249776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf78389-fcda-41c4-94ea-ecf34efbf53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.250413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf78389-fcda-41c4-94ea-ecf34efbf53a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.250688 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf78389-fcda-41c4-94ea-ecf34efbf53a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.250963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf78389-fcda-41c4-94ea-ecf34efbf53a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.259290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf78389-fcda-41c4-94ea-ecf34efbf53a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.260770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf78389-fcda-41c4-94ea-ecf34efbf53a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.264471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-eaf78389-fcda-41c4-94ea-ecf34efbf53a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:00:57.275000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:57.276144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 297/588] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:57.277065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf78389-fcda-41c4-94ea-ecf34efbf53a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.299888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f721ac9-2a39-4ef9-8da6-3908e6384932 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.336744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97747f53-c15e-4f14-a8b8-6e4e9f4208ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.337470 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97747f53-c15e-4f14-a8b8-6e4e9f4208ae None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.337653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97747f53-c15e-4f14-a8b8-6e4e9f4208ae None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.337886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97747f53-c15e-4f14-a8b8-6e4e9f4208ae None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.338532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97747f53-c15e-4f14-a8b8-6e4e9f4208ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:57.338834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97747f53-c15e-4f14-a8b8-6e4e9f4208ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.340248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97747f53-c15e-4f14-a8b8-6e4e9f4208ae None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.353081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 292/589] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:57.353415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97747f53-c15e-4f14-a8b8-6e4e9f4208ae None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.389010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf78389-fcda-41c4-94ea-ecf34efbf53a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.398638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:57.424371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c8a0e-38ca-4677-b7ca-e5a9f4fa2876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.425008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c8a0e-38ca-4677-b7ca-e5a9f4fa2876 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.425286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c8a0e-38ca-4677-b7ca-e5a9f4fa2876 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.425560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875c8a0e-38ca-4677-b7ca-e5a9f4fa2876 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.436329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c8a0e-38ca-4677-b7ca-e5a9f4fa2876 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.436329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875c8a0e-38ca-4677-b7ca-e5a9f4fa2876 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-875c8a0e-38ca-4677-b7ca-e5a9f4fa2876 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:00:57.448701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:57.449884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:57.451087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:57.451087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:57.451087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:00:57.451087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:00:57.451087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:00:57.451087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:00:57.451087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:00:57.451087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:00:57.451087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 14:00:57.451087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:57.451961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 298/590] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:57.452807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c8a0e-38ca-4677-b7ca-e5a9f4fa2876 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.470846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97747f53-c15e-4f14-a8b8-6e4e9f4208ae None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.508930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ee99a-f2d6-46fa-ac79-57838ce4a117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.509563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ee99a-f2d6-46fa-ac79-57838ce4a117 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.509840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ee99a-f2d6-46fa-ac79-57838ce4a117 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.510108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ee99a-f2d6-46fa-ac79-57838ce4a117 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.510795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ee99a-f2d6-46fa-ac79-57838ce4a117 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:57.511099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ee99a-f2d6-46fa-ac79-57838ce4a117 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.512599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68ee99a-f2d6-46fa-ac79-57838ce4a117 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.522499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:57.529944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 293/591] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:57.529944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ee99a-f2d6-46fa-ac79-57838ce4a117 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.570538 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875c8a0e-38ca-4677-b7ca-e5a9f4fa2876 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.606240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae13390-1b5c-4a62-9908-cd64481f40a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.606937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae13390-1b5c-4a62-9908-cd64481f40a2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.607285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae13390-1b5c-4a62-9908-cd64481f40a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.607648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae13390-1b5c-4a62-9908-cd64481f40a2 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.608438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae13390-1b5c-4a62-9908-cd64481f40a2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:57.608879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae13390-1b5c-4a62-9908-cd64481f40a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae13390-1b5c-4a62-9908-cd64481f40a2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.633771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 299/592] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:57.633771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae13390-1b5c-4a62-9908-cd64481f40a2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.637358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ee99a-f2d6-46fa-ac79-57838ce4a117 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.639093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:57.673131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218405b-6abd-4907-95d3-beaddb3ba871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.673913 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218405b-6abd-4907-95d3-beaddb3ba871 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.674271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218405b-6abd-4907-95d3-beaddb3ba871 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.674618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4218405b-6abd-4907-95d3-beaddb3ba871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.682567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218405b-6abd-4907-95d3-beaddb3ba871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.684262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4218405b-6abd-4907-95d3-beaddb3ba871 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-4218405b-6abd-4907-95d3-beaddb3ba871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:00:57.697746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:57.698901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:57.700053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:00:57.700053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:00:57.700053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:00:57.700053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:00:57.700053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:00:57.700053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:00:57.700053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:57.700053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:57.706151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 294/593] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:57.707204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218405b-6abd-4907-95d3-beaddb3ba871 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.744761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae13390-1b5c-4a62-9908-cd64481f40a2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.762350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:57.782290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70993930-bbc4-43a7-9de9-900f5b77dec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.782904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70993930-bbc4-43a7-9de9-900f5b77dec3 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.783183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70993930-bbc4-43a7-9de9-900f5b77dec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.783473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70993930-bbc4-43a7-9de9-900f5b77dec3 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.784221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70993930-bbc4-43a7-9de9-900f5b77dec3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:57.784531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70993930-bbc4-43a7-9de9-900f5b77dec3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.786255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70993930-bbc4-43a7-9de9-900f5b77dec3 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.806262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 300/594] 10.222.0.26 () {56 vars in 1139 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:57.807314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70993930-bbc4-43a7-9de9-900f5b77dec3 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.817617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4218405b-6abd-4907-95d3-beaddb3ba871 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.851848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc35c7-8c21-4cd0-800b-2dc18144e4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.852727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc35c7-8c21-4cd0-800b-2dc18144e4ff None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.852998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc35c7-8c21-4cd0-800b-2dc18144e4ff None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.853270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc35c7-8c21-4cd0-800b-2dc18144e4ff None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/cb24a1b55ee7408ab24c68991aec54ec/roles/a24499651a7b4225ac5459b8b59f1e24` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.867877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc35c7-8c21-4cd0-800b-2dc18144e4ff None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=cb24a1b55ee7408ab24c68991aec54ec, role_id=a24499651a7b4225ac5459b8b59f1e24)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.869322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc35c7-8c21-4cd0-800b-2dc18144e4ff None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.882067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 295/595] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:00:57 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/cb24a1b55ee7408ab24c68991aec54ec/roles/a24499651a7b4225ac5459b8b59f1e24 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:57.882862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc35c7-8c21-4cd0-800b-2dc18144e4ff None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:57.897781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:57.918506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70993930-bbc4-43a7-9de9-900f5b77dec3 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:57.956944 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9658ee-fee2-443b-932e-4be82f21a04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:57.957594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9658ee-fee2-443b-932e-4be82f21a04e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:57.958039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9658ee-fee2-443b-932e-4be82f21a04e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:57.958317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9658ee-fee2-443b-932e-4be82f21a04e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:57.969545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9658ee-fee2-443b-932e-4be82f21a04e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:57.971279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9658ee-fee2-443b-932e-4be82f21a04e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-dc9658ee-fee2-443b-932e-4be82f21a04e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:57.988733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:57.989932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:57.995742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:57.995742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:57.995742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:57.995742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 301/596] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:57.995742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9658ee-fee2-443b-932e-4be82f21a04e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:58.008310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc35c7-8c21-4cd0-800b-2dc18144e4ff None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:58.028331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:58.046238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f653d73-bedb-4375-a6ce-b69641922d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:58.046855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f653d73-bedb-4375-a6ce-b69641922d4e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.047134 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f653d73-bedb-4375-a6ce-b69641922d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.047403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f653d73-bedb-4375-a6ce-b69641922d4e None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.048156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f653d73-bedb-4375-a6ce-b69641922d4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:58.048457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f653d73-bedb-4375-a6ce-b69641922d4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:58.049912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f653d73-bedb-4375-a6ce-b69641922d4e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:58.068724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 296/597] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:58.070270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2063-e739-4517-a710-e1cdaa6fcbb2 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.070615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2063-e739-4517-a710-e1cdaa6fcbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.070949 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc2063-e739-4517-a710-e1cdaa6fcbb2 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.072492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 297/598] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:00:58.073299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc2063-e739-4517-a710-e1cdaa6fcbb2 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:58.112470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9658ee-fee2-443b-932e-4be82f21a04e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:58.149730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd65872-5733-48c8-9e47-689c31ed33bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:58.150271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd65872-5733-48c8-9e47-689c31ed33bc None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.150545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd65872-5733-48c8-9e47-689c31ed33bc None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.150813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd65872-5733-48c8-9e47-689c31ed33bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.153395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:58.163461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd65872-5733-48c8-9e47-689c31ed33bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:58.164937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd65872-5733-48c8-9e47-689c31ed33bc None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-afd65872-5733-48c8-9e47-689c31ed33bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:58.180219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:58.181196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:58.182104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:58.182104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:58.182104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:58.182104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 302/599] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:58.182104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd65872-5733-48c8-9e47-689c31ed33bc None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:58.193289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc2063-e739-4517-a710-e1cdaa6fcbb2 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:58.229815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a553a20-d63f-402a-b702-364265d3037c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:58.230432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a553a20-d63f-402a-b702-364265d3037c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.230707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a553a20-d63f-402a-b702-364265d3037c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.230976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a553a20-d63f-402a-b702-364265d3037c None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.231647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a553a20-d63f-402a-b702-364265d3037c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:58.231950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a553a20-d63f-402a-b702-364265d3037c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:58.233333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a553a20-d63f-402a-b702-364265d3037c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:58.249994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 298/600] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:58.251214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96affb-faeb-4783-9798-9b552ec3f3f0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.251528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96affb-faeb-4783-9798-9b552ec3f3f0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.251803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c96affb-faeb-4783-9798-9b552ec3f3f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.270751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4c96affb-faeb-4783-9798-9b552ec3f3f0 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:00:58.271970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4c96affb-faeb-4783-9798-9b552ec3f3f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:00:58.278076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:58.300175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd65872-5733-48c8-9e47-689c31ed33bc None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:58.338292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e8d0d-1370-4fc0-883b-bf9bc9dda542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:58.338945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e8d0d-1370-4fc0-883b-bf9bc9dda542 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.339220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e8d0d-1370-4fc0-883b-bf9bc9dda542 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.339559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776e8d0d-1370-4fc0-883b-bf9bc9dda542 None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/cf75b583add44b6b87bff5dc67080d1c/roles/a24499651a7b4225ac5459b8b59f1e24` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.343498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 299/601] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:00:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:00:58.343498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96affb-faeb-4783-9798-9b552ec3f3f0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:58.363362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e8d0d-1370-4fc0-883b-bf9bc9dda542 None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=cf75b583add44b6b87bff5dc67080d1c, role_id=a24499651a7b4225ac5459b8b59f1e24)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:58.365944 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776e8d0d-1370-4fc0-883b-bf9bc9dda542 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:58.381723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 303/602] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:00:58 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/cf75b583add44b6b87bff5dc67080d1c/roles/a24499651a7b4225ac5459b8b59f1e24 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:58.382694 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e8d0d-1370-4fc0-883b-bf9bc9dda542 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:58.421371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:58.479301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c96affb-faeb-4783-9798-9b552ec3f3f0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:58.503188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776e8d0d-1370-4fc0-883b-bf9bc9dda542 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:58.523627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343151b4-65f5-450f-9fb3-ded38ad5a755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:58.524957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343151b4-65f5-450f-9fb3-ded38ad5a755 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.525398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343151b4-65f5-450f-9fb3-ded38ad5a755 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.525817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343151b4-65f5-450f-9fb3-ded38ad5a755 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.526653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343151b4-65f5-450f-9fb3-ded38ad5a755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:58.527108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343151b4-65f5-450f-9fb3-ded38ad5a755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:58.529398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343151b4-65f5-450f-9fb3-ded38ad5a755 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:58.538636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:58.547759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606f966-4554-46ec-af7b-5120677fabf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:58.548571 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f966-4554-46ec-af7b-5120677fabf2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.548953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f966-4554-46ec-af7b-5120677fabf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.549325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606f966-4554-46ec-af7b-5120677fabf2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.550364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 300/603] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:58.556921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343151b4-65f5-450f-9fb3-ded38ad5a755 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:58.560051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606f966-4554-46ec-af7b-5120677fabf2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:58.562793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606f966-4554-46ec-af7b-5120677fabf2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-c606f966-4554-46ec-af7b-5120677fabf2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:00:58.576998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:58.578123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:58.579201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:00:58.579201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:00:58.579201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:00:58.579201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:00:58.579201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:00:58.579201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:00:58.579201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:58.579201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:00:58.580975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 304/604] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:58.589523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606f966-4554-46ec-af7b-5120677fabf2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:58.721384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343151b4-65f5-450f-9fb3-ded38ad5a755 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:58.751037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:58.761814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606f966-4554-46ec-af7b-5120677fabf2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:58.767859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aee217-7165-4af8-af8b-e83a5e69e8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:58.769581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aee217-7165-4af8-af8b-e83a5e69e8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.769581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aee217-7165-4af8-af8b-e83a5e69e8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.769581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aee217-7165-4af8-af8b-e83a5e69e8c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.789001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aee217-7165-4af8-af8b-e83a5e69e8c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:58.790752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aee217-7165-4af8-af8b-e83a5e69e8c4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:58.802720 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66887bd-2785-45e4-a5bf-1b0347cb02a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:58.803619 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66887bd-2785-45e4-a5bf-1b0347cb02a0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a7aee217-7165-4af8-af8b-e83a5e69e8c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:00:58.803974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:58.805003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:58.806077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:00:58.806077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:00:58.806077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:00:58.806077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:00:58.806077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:00:58.806077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:00:58.806077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:00:58.806077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:58.806785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66887bd-2785-45e4-a5bf-1b0347cb02a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.807263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66887bd-2785-45e4-a5bf-1b0347cb02a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.807578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 301/605] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:00:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:00:58.808334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66887bd-2785-45e4-a5bf-1b0347cb02a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:58.808993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aee217-7165-4af8-af8b-e83a5e69e8c4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:58.813875 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66887bd-2785-45e4-a5bf-1b0347cb02a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:58.815430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66887bd-2785-45e4-a5bf-1b0347cb02a0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:58.830073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 305/606] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:00:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:58.835736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66887bd-2785-45e4-a5bf-1b0347cb02a0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:58.890402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:58.925505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aee217-7165-4af8-af8b-e83a5e69e8c4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:58.944036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66887bd-2785-45e4-a5bf-1b0347cb02a0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:58.964237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49eb2a-b4e5-4a2b-af18-2c8ee8dd4e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:58.965294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49eb2a-b4e5-4a2b-af18-2c8ee8dd4e46 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.965753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49eb2a-b4e5-4a2b-af18-2c8ee8dd4e46 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.966255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49eb2a-b4e5-4a2b-af18-2c8ee8dd4e46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.978922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49eb2a-b4e5-4a2b-af18-2c8ee8dd4e46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:00:58.980334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a73a9-7104-4df5-8c51-bbda011bd761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:58.980937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a73a9-7104-4df5-8c51-bbda011bd761 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:58.981214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a73a9-7104-4df5-8c51-bbda011bd761 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:58.981488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a73a9-7104-4df5-8c51-bbda011bd761 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:58.982178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a73a9-7104-4df5-8c51-bbda011bd761 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:58.982484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a73a9-7104-4df5-8c51-bbda011bd761 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:58.983952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508a73a9-7104-4df5-8c51-bbda011bd761 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6f49eb2a-b4e5-4a2b-af18-2c8ee8dd4e46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:00:58.986432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:58.987361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:00:58.988486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:58.995645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 302/607] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:00:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:58.996745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49eb2a-b4e5-4a2b-af18-2c8ee8dd4e46 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:59.001222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 306/608] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:00:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:59.001737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a73a9-7104-4df5-8c51-bbda011bd761 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:59.045839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:59.109096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49eb2a-b4e5-4a2b-af18-2c8ee8dd4e46 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:59.109347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a73a9-7104-4df5-8c51-bbda011bd761 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:59.123824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:59.146943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47ac06-ae5e-42cc-9baa-ecfe201bab2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:59.147667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47ac06-ae5e-42cc-9baa-ecfe201bab2b None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:59.147969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47ac06-ae5e-42cc-9baa-ecfe201bab2b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:59.148246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47ac06-ae5e-42cc-9baa-ecfe201bab2b None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/fdc278f50e794e12a27d30fca9d131c1/roles/a24499651a7b4225ac5459b8b59f1e24` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:59.152140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eba25d-7784-4b3e-b40b-75482acdfa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:59.152989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba25d-7784-4b3e-b40b-75482acdfa9f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:59.153487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba25d-7784-4b3e-b40b-75482acdfa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:59.153955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba25d-7784-4b3e-b40b-75482acdfa9f None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:59.154826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eba25d-7784-4b3e-b40b-75482acdfa9f None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:59.155326 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eba25d-7784-4b3e-b40b-75482acdfa9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:59.157038 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eba25d-7784-4b3e-b40b-75482acdfa9f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:59.163770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47ac06-ae5e-42cc-9baa-ecfe201bab2b None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=fdc278f50e794e12a27d30fca9d131c1, role_id=a24499651a7b4225ac5459b8b59f1e24)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:59.165230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c47ac06-ae5e-42cc-9baa-ecfe201bab2b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:59.167581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 307/609] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:00:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:59.168749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eba25d-7784-4b3e-b40b-75482acdfa9f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:59.191565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 303/610] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:00:58 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/fdc278f50e794e12a27d30fca9d131c1/roles/a24499651a7b4225ac5459b8b59f1e24 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:59.192721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47ac06-ae5e-42cc-9baa-ecfe201bab2b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:59.297756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eba25d-7784-4b3e-b40b-75482acdfa9f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:59.369642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761d3dd-5063-49bf-b74d-6ac76435d979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:59.370318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761d3dd-5063-49bf-b74d-6ac76435d979 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:59.370596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761d3dd-5063-49bf-b74d-6ac76435d979 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:59.370867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761d3dd-5063-49bf-b74d-6ac76435d979 None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/12c71caf31354ec193e3e2da231f1e77/roles/a24499651a7b4225ac5459b8b59f1e24` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:59.390145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:59.407666 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47ac06-ae5e-42cc-9baa-ecfe201bab2b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:59.409371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761d3dd-5063-49bf-b74d-6ac76435d979 None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=12c71caf31354ec193e3e2da231f1e77, role_id=a24499651a7b4225ac5459b8b59f1e24)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:59.411131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761d3dd-5063-49bf-b74d-6ac76435d979 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:59.435746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 308/611] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:00:59 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/12c71caf31354ec193e3e2da231f1e77/roles/a24499651a7b4225ac5459b8b59f1e24 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:00:59.450736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44af35-f2fb-4c2e-9b07-52c13fc12f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:59.451487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44af35-f2fb-4c2e-9b07-52c13fc12f9d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:59.452049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44af35-f2fb-4c2e-9b07-52c13fc12f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:59.452370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb44af35-f2fb-4c2e-9b07-52c13fc12f9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:59.469062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44af35-f2fb-4c2e-9b07-52c13fc12f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:59.470822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb44af35-f2fb-4c2e-9b07-52c13fc12f9d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-eb44af35-f2fb-4c2e-9b07-52c13fc12f9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:00:59.488948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:59.490109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:00:59.491019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:00:59.491019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:00:59.491019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:59.491019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 304/612] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:00:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:00:59.496383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761d3dd-5063-49bf-b74d-6ac76435d979 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:59.530498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:00:59.628744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761d3dd-5063-49bf-b74d-6ac76435d979 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:59.675466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616ffdc-f8af-46bd-8444-c2e637d4da88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:59.676221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616ffdc-f8af-46bd-8444-c2e637d4da88 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:59.676558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616ffdc-f8af-46bd-8444-c2e637d4da88 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:59.676902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616ffdc-f8af-46bd-8444-c2e637d4da88 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:59.677620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616ffdc-f8af-46bd-8444-c2e637d4da88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:00:59.678012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616ffdc-f8af-46bd-8444-c2e637d4da88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:59.679551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9616ffdc-f8af-46bd-8444-c2e637d4da88 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:59.692604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 309/613] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:00:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:00:59.698007 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44af35-f2fb-4c2e-9b07-52c13fc12f9d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:59.810002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb44af35-f2fb-4c2e-9b07-52c13fc12f9d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:00:59.845269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209a20e-ce1b-4cce-b4d9-d836073ce29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:00:59.846046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209a20e-ce1b-4cce-b4d9-d836073ce29f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:00:59.846423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209a20e-ce1b-4cce-b4d9-d836073ce29f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:00:59.846795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8209a20e-ce1b-4cce-b4d9-d836073ce29f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:00:59.854724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209a20e-ce1b-4cce-b4d9-d836073ce29f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:00:59.856323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8209a20e-ce1b-4cce-b4d9-d836073ce29f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-8209a20e-ce1b-4cce-b4d9-d836073ce29f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:00:59.868517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:00:59.869695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:00:59.870846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:00:59.870846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:00:59.870846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:00:59.870846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:00:59.870846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:00:59.870846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:00:59.870846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:00:59.870846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:00:59.870846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:00:59.870846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:00:59.872621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 305/614] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:00:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:00:59.880640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616ffdc-f8af-46bd-8444-c2e637d4da88 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:00:59.987230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:00.013759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616ffdc-f8af-46bd-8444-c2e637d4da88 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:00.055518 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a329ab-1a8c-4b07-9e35-8228bcea3ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:00.056129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a329ab-1a8c-4b07-9e35-8228bcea3ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:00.056407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a329ab-1a8c-4b07-9e35-8228bcea3ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:00.056678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a329ab-1a8c-4b07-9e35-8228bcea3ff6 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:00.057398 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a329ab-1a8c-4b07-9e35-8228bcea3ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:00.057699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a329ab-1a8c-4b07-9e35-8228bcea3ff6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:00.059110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a329ab-1a8c-4b07-9e35-8228bcea3ff6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:00.076526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 310/615] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 10 14:00:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:00.084587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209a20e-ce1b-4cce-b4d9-d836073ce29f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:00.088523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:00.231197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8209a20e-ce1b-4cce-b4d9-d836073ce29f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:00.275139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c112243-d360-4a3e-b9be-071ef6b355d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:00.275802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112243-d360-4a3e-b9be-071ef6b355d5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:00.276079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112243-d360-4a3e-b9be-071ef6b355d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:00.276347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c112243-d360-4a3e-b9be-071ef6b355d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:00.288129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c112243-d360-4a3e-b9be-071ef6b355d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:00.289593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c112243-d360-4a3e-b9be-071ef6b355d5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9c112243-d360-4a3e-b9be-071ef6b355d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:00.304482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:00.305648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:00.306746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:00.306746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:00.306746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:00.306746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 306/616] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:00.312726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a329ab-1a8c-4b07-9e35-8228bcea3ff6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:00.324411 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43572e9a-6afa-4431-a7ff-5eed9e38631b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:00.326462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43572e9a-6afa-4431-a7ff-5eed9e38631b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:00.326888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43572e9a-6afa-4431-a7ff-5eed9e38631b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:00.328747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 307/617] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:00.336557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caa313-7e46-4cbc-8d46-4fb0e32e9f6c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:00.337054 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caa313-7e46-4cbc-8d46-4fb0e32e9f6c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:00.337478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caa313-7e46-4cbc-8d46-4fb0e32e9f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:00.365381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a6caa313-7e46-4cbc-8d46-4fb0e32e9f6c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:00.366871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a6caa313-7e46-4cbc-8d46-4fb0e32e9f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:00.440697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a329ab-1a8c-4b07-9e35-8228bcea3ff6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:00.475816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:00.479205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d329ae-cb1e-455d-973b-a0ef453e3d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:00.480042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d329ae-cb1e-455d-973b-a0ef453e3d00 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:00.480330 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d329ae-cb1e-455d-973b-a0ef453e3d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:00.480713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d329ae-cb1e-455d-973b-a0ef453e3d00 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:00.481590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d329ae-cb1e-455d-973b-a0ef453e3d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:00.481928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d329ae-cb1e-455d-973b-a0ef453e3d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:00.483400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d329ae-cb1e-455d-973b-a0ef453e3d00 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:00.492831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:00.497707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 311/618] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:00.497855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 308/619] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:00.507504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caa313-7e46-4cbc-8d46-4fb0e32e9f6c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:00.619477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caa313-7e46-4cbc-8d46-4fb0e32e9f6c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:00.654290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387fcde-7733-4411-b13a-37897f61e2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:00.654926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387fcde-7733-4411-b13a-37897f61e2da None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:00.655203 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387fcde-7733-4411-b13a-37897f61e2da None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:00.655540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387fcde-7733-4411-b13a-37897f61e2da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:00.662929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387fcde-7733-4411-b13a-37897f61e2da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:00.664405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1387fcde-7733-4411-b13a-37897f61e2da None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-1387fcde-7733-4411-b13a-37897f61e2da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:00.676524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:00.677641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:00.678504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:00.678504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:00.678504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:00.678504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:00.678504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:00.678504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:00.678504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:00.678504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:00.678504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 309/620] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:00.683835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d329ae-cb1e-455d-973b-a0ef453e3d00 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:00.805626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d329ae-cb1e-455d-973b-a0ef453e3d00 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:00.838665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2faed6-9498-4dc4-a451-3f1a0d0a0be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:00.839269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2faed6-9498-4dc4-a451-3f1a0d0a0be8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:00.839579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2faed6-9498-4dc4-a451-3f1a0d0a0be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:00.839853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2faed6-9498-4dc4-a451-3f1a0d0a0be8 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:00.840537 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2faed6-9498-4dc4-a451-3f1a0d0a0be8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:00.840839 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2faed6-9498-4dc4-a451-3f1a0d0a0be8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:00.842322 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2faed6-9498-4dc4-a451-3f1a0d0a0be8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:00.854852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 312/621] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:00.860583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387fcde-7733-4411-b13a-37897f61e2da None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:00.897673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2faed6-9498-4dc4-a451-3f1a0d0a0be8 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:00.928949 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:00.965671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:01.030968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1387fcde-7733-4411-b13a-37897f61e2da None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:01.038159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2faed6-9498-4dc4-a451-3f1a0d0a0be8 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:01.108231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df667c-a38e-4866-bce4-a950d6191b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:01.109034 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df667c-a38e-4866-bce4-a950d6191b3e None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:01.109313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df667c-a38e-4866-bce4-a950d6191b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:01.109588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6df667c-a38e-4866-bce4-a950d6191b3e None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/bc0a7e1b772b4703ac2e5ec33b5ecf6b/roles/a24499651a7b4225ac5459b8b59f1e24` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:01.116072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cbb7c1-ed79-43ba-a727-b2c7bb215f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:01.116707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbb7c1-ed79-43ba-a727-b2c7bb215f11 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:01.116985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbb7c1-ed79-43ba-a727-b2c7bb215f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:01.117259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cbb7c1-ed79-43ba-a727-b2c7bb215f11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:01.131900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df667c-a38e-4866-bce4-a950d6191b3e None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=bc0a7e1b772b4703ac2e5ec33b5ecf6b, role_id=a24499651a7b4225ac5459b8b59f1e24)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:01.133360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6df667c-a38e-4866-bce4-a950d6191b3e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:01.143327 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cbb7c1-ed79-43ba-a727-b2c7bb215f11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-57cbb7c1-ed79-43ba-a727-b2c7bb215f11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:01.149036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:01.149970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:01.150769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:01.151915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 310/622] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:01:00 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/bc0a7e1b772b4703ac2e5ec33b5ecf6b/roles/a24499651a7b4225ac5459b8b59f1e24 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:01:01.153503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 313/623] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:01:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:01.159500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df667c-a38e-4866-bce4-a950d6191b3e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:01.234870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94081e35-ff74-451a-8ad0-cd53a71e891e None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:01.235562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94081e35-ff74-451a-8ad0-cd53a71e891e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:01.236057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94081e35-ff74-451a-8ad0-cd53a71e891e None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:01.237927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 314/624] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:01.248936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8192e3-183f-4f4b-a976-eb484aea37f9 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:01.249556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8192e3-183f-4f4b-a976-eb484aea37f9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:01.250024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8192e3-183f-4f4b-a976-eb484aea37f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:01.270417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-ca8192e3-183f-4f4b-a976-eb484aea37f9 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:01.272120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ca8192e3-183f-4f4b-a976-eb484aea37f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:01.300515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:01.309914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6df667c-a38e-4866-bce4-a950d6191b3e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:01.353309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5868df-5077-4217-9bce-3a62cfb1f3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:01.353924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5868df-5077-4217-9bce-3a62cfb1f3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:01.354213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5868df-5077-4217-9bce-3a62cfb1f3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:01.354478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5868df-5077-4217-9bce-3a62cfb1f3f9 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:01.355183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5868df-5077-4217-9bce-3a62cfb1f3f9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:01.355391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 315/625] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:01:01.364067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5868df-5077-4217-9bce-3a62cfb1f3f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:01.364067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5868df-5077-4217-9bce-3a62cfb1f3f9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:01.381649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 311/626] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:01:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:01.384954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8192e3-183f-4f4b-a976-eb484aea37f9 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:01.456494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:01.506723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8192e3-183f-4f4b-a976-eb484aea37f9 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:01.541707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44f8f1-41a6-4bf6-b752-e64a15d2ed01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:01.542342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f8f1-41a6-4bf6-b752-e64a15d2ed01 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:01.542616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f8f1-41a6-4bf6-b752-e64a15d2ed01 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:01.542884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e44f8f1-41a6-4bf6-b752-e64a15d2ed01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:01.554177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44f8f1-41a6-4bf6-b752-e64a15d2ed01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:01.555713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e44f8f1-41a6-4bf6-b752-e64a15d2ed01 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-9e44f8f1-41a6-4bf6-b752-e64a15d2ed01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:01.571002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:01.571966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:01.572998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:01.572998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:01.572998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:01.572998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 316/627] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:01.579211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5868df-5077-4217-9bce-3a62cfb1f3f9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:01.691364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5868df-5077-4217-9bce-3a62cfb1f3f9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:01.734012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44f8f1-41a6-4bf6-b752-e64a15d2ed01 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:01.749391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816f7eb-cae8-4109-9147-82c6057a7f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:01.749997 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816f7eb-cae8-4109-9147-82c6057a7f50 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:01.750336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816f7eb-cae8-4109-9147-82c6057a7f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:01.750610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e816f7eb-cae8-4109-9147-82c6057a7f50 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:01.751413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816f7eb-cae8-4109-9147-82c6057a7f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:01.751751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816f7eb-cae8-4109-9147-82c6057a7f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:01.753548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e816f7eb-cae8-4109-9147-82c6057a7f50 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:01.780731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 312/628] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:01.787311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816f7eb-cae8-4109-9147-82c6057a7f50 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:01.823921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:01.942755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e44f8f1-41a6-4bf6-b752-e64a15d2ed01 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:01.946794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:01.992514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e816f7eb-cae8-4109-9147-82c6057a7f50 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:01.998625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20dc34-a29f-46ee-8777-55984ad0b8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:01.999262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20dc34-a29f-46ee-8777-55984ad0b8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:01.999583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20dc34-a29f-46ee-8777-55984ad0b8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:01.999861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20dc34-a29f-46ee-8777-55984ad0b8f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.010808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20dc34-a29f-46ee-8777-55984ad0b8f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-0a20dc34-a29f-46ee-8777-55984ad0b8f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:02.016182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:02.017176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:02.018009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 317/629] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:01:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:02.019144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a70dd-12b2-48ff-afdd-6ab0308dba29 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.019144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a70dd-12b2-48ff-afdd-6ab0308dba29 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.019306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a70dd-12b2-48ff-afdd-6ab0308dba29 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.024863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 318/630] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:02.026068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c128c-0c14-4799-9a02-031982db564b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.026348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c128c-0c14-4799-9a02-031982db564b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.026621 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99c128c-0c14-4799-9a02-031982db564b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.044421 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a656a-9989-4f70-961c-a18ba4b6aa17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:02.045194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a656a-9989-4f70-961c-a18ba4b6aa17 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.045418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c99c128c-0c14-4799-9a02-031982db564b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:02.045595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a656a-9989-4f70-961c-a18ba4b6aa17 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.045745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a656a-9989-4f70-961c-a18ba4b6aa17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.046615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c99c128c-0c14-4799-9a02-031982db564b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:02.055363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a656a-9989-4f70-961c-a18ba4b6aa17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:02.057313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9a656a-9989-4f70-961c-a18ba4b6aa17 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-4c9a656a-9989-4f70-961c-a18ba4b6aa17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:02.074130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:02.076110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:02.077355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:02.077355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:02.077355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:02.077355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:02.077355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:02.077355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:02.077355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:02.077355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:02.077355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:01:02.077355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:02.080581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 313/631] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 10 14:01:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:02.080581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a656a-9989-4f70-961c-a18ba4b6aa17 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:02.131580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 319/632] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:02.132543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c128c-0c14-4799-9a02-031982db564b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:02.182391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:02.209831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a656a-9989-4f70-961c-a18ba4b6aa17 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:02.247889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24590e32-4e7f-4f7c-99d1-5483d6d40c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:02.248492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24590e32-4e7f-4f7c-99d1-5483d6d40c5d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.248769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24590e32-4e7f-4f7c-99d1-5483d6d40c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.249041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24590e32-4e7f-4f7c-99d1-5483d6d40c5d None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.249710 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24590e32-4e7f-4f7c-99d1-5483d6d40c5d None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:02.250121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24590e32-4e7f-4f7c-99d1-5483d6d40c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:02.251525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24590e32-4e7f-4f7c-99d1-5483d6d40c5d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:02.263695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 314/633] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:01:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:02.266738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24590e32-4e7f-4f7c-99d1-5483d6d40c5d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:02.307076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:02.319122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99c128c-0c14-4799-9a02-031982db564b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:02.402901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05de6702-2d5a-4765-baac-1e8710a32d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:02.405699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de6702-2d5a-4765-baac-1e8710a32d87 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.406842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de6702-2d5a-4765-baac-1e8710a32d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.409051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05de6702-2d5a-4765-baac-1e8710a32d87 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.410002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de6702-2d5a-4765-baac-1e8710a32d87 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:02.410841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de6702-2d5a-4765-baac-1e8710a32d87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:02.411426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24590e32-4e7f-4f7c-99d1-5483d6d40c5d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:02.413096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05de6702-2d5a-4765-baac-1e8710a32d87 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:02.453462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 320/634] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 10 14:01:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:02.454788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011d88c-d688-419f-a394-dd7ed090794b None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.455076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011d88c-d688-419f-a394-dd7ed090794b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.455350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011d88c-d688-419f-a394-dd7ed090794b None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.456814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 321/635] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:02.460570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011d88c-d688-419f-a394-dd7ed090794b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:02.462332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aab33df-362a-49bf-b9ec-2a3876840e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:02.462970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab33df-362a-49bf-b9ec-2a3876840e72 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.463246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab33df-362a-49bf-b9ec-2a3876840e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.463546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aab33df-362a-49bf-b9ec-2a3876840e72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.473477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:02.476367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab33df-362a-49bf-b9ec-2a3876840e72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:02.477998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aab33df-362a-49bf-b9ec-2a3876840e72 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6aab33df-362a-49bf-b9ec-2a3876840e72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:02.493945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:02.494961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:02.495875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:02.495875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:02.495875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:02.499523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 315/636] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:02.501300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4e76-7d83-482f-b056-863b6d912735 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.501700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4e76-7d83-482f-b056-863b6d912735 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.502080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31a4e76-7d83-482f-b056-863b6d912735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.520670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d31a4e76-7d83-482f-b056-863b6d912735 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:02.521853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d31a4e76-7d83-482f-b056-863b6d912735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:02.577449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011d88c-d688-419f-a394-dd7ed090794b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:02.593187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 316/637] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:02.593802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a4e76-7d83-482f-b056-863b6d912735 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:02.615194 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df3146-6231-4492-b694-1dc63ed194ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:02.615833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df3146-6231-4492-b694-1dc63ed194ae None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.616108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df3146-6231-4492-b694-1dc63ed194ae None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.616381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df3146-6231-4492-b694-1dc63ed194ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.640667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:02.699614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df3146-6231-4492-b694-1dc63ed194ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:02.701067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df3146-6231-4492-b694-1dc63ed194ae None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:02.736254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31a4e76-7d83-482f-b056-863b6d912735 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:02.761836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:02.776960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446cd15-8ad8-4800-9485-d88a1132af0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:02.777591 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446cd15-8ad8-4800-9485-d88a1132af0a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.777867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446cd15-8ad8-4800-9485-d88a1132af0a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.778138 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446cd15-8ad8-4800-9485-d88a1132af0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.789940 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446cd15-8ad8-4800-9485-d88a1132af0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-7446cd15-8ad8-4800-9485-d88a1132af0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:02.795192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:02.796160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:02.796996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 317/638] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:01:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:02.797668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446cd15-8ad8-4800-9485-d88a1132af0a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:02.826279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 322/639] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:01:02.827033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df3146-6231-4492-b694-1dc63ed194ae None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:02.884714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:02.928062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446cd15-8ad8-4800-9485-d88a1132af0a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:02.941458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df3146-6231-4492-b694-1dc63ed194ae None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:02.964053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2f67d-4d4c-450e-a9ce-295ca6ece7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:02.964651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2f67d-4d4c-450e-a9ce-295ca6ece7ef None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.964926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2f67d-4d4c-450e-a9ce-295ca6ece7ef None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.965195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb2f67d-4d4c-450e-a9ce-295ca6ece7ef None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.965826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2f67d-4d4c-450e-a9ce-295ca6ece7ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:02.966124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2f67d-4d4c-450e-a9ce-295ca6ece7ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:02.967528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb2f67d-4d4c-450e-a9ce-295ca6ece7ef None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:02.978513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecfeac-6b58-4097-9fec-f2f16ac805d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:02.983555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 318/640] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:02.983689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecfeac-6b58-4097-9fec-f2f16ac805d2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:02.983689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecfeac-6b58-4097-9fec-f2f16ac805d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:02.983689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fecfeac-6b58-4097-9fec-f2f16ac805d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:02.983948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2f67d-4d4c-450e-a9ce-295ca6ece7ef None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:02.984023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecfeac-6b58-4097-9fec-f2f16ac805d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:02.984023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecfeac-6b58-4097-9fec-f2f16ac805d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:02.984023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fecfeac-6b58-4097-9fec-f2f16ac805d2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:02.995561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 323/641] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:01:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:02.995561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecfeac-6b58-4097-9fec-f2f16ac805d2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.029927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:03.087111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb2f67d-4d4c-450e-a9ce-295ca6ece7ef None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.105838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fecfeac-6b58-4097-9fec-f2f16ac805d2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.119907 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:03.120736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1de79d-801a-44ae-ba11-738df4a9d65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.121050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1de79d-801a-44ae-ba11-738df4a9d65a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.121327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1de79d-801a-44ae-ba11-738df4a9d65a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.121599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1de79d-801a-44ae-ba11-738df4a9d65a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.132824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1de79d-801a-44ae-ba11-738df4a9d65a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:03.134309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1de79d-801a-44ae-ba11-738df4a9d65a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:03.144702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50075f-365c-40a2-89c3-58cd36a04a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.145805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50075f-365c-40a2-89c3-58cd36a04a4c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.146261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50075f-365c-40a2-89c3-58cd36a04a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.146763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50075f-365c-40a2-89c3-58cd36a04a4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-7e1de79d-801a-44ae-ba11-738df4a9d65a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:03.148237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:03.149130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:03.149989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:03.149989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:03.149989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:03.149989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 319/642] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:03.150216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1de79d-801a-44ae-ba11-738df4a9d65a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.163958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50075f-365c-40a2-89c3-58cd36a04a4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-5f50075f-365c-40a2-89c3-58cd36a04a4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:03.169500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:03.170477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:03.171316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 324/643] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:01:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:03.176528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50075f-365c-40a2-89c3-58cd36a04a4c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.250469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:03.260990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1de79d-801a-44ae-ba11-738df4a9d65a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.286707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50075f-365c-40a2-89c3-58cd36a04a4c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.294228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072aee4-a142-4b41-b1ab-8db9eae15125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.294856 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072aee4-a142-4b41-b1ab-8db9eae15125 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.295131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072aee4-a142-4b41-b1ab-8db9eae15125 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.295405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072aee4-a142-4b41-b1ab-8db9eae15125 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.302978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072aee4-a142-4b41-b1ab-8db9eae15125 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:03.304479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072aee4-a142-4b41-b1ab-8db9eae15125 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d072aee4-a142-4b41-b1ab-8db9eae15125 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:03.315602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:03.317151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 320/644] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:01:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:03.318635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072aee4-a142-4b41-b1ab-8db9eae15125 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.327500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455efbc8-70fc-4735-92fa-89417acace8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.328394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455efbc8-70fc-4735-92fa-89417acace8c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.328859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455efbc8-70fc-4735-92fa-89417acace8c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.329649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455efbc8-70fc-4735-92fa-89417acace8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.342648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455efbc8-70fc-4735-92fa-89417acace8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:03.344451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455efbc8-70fc-4735-92fa-89417acace8c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-455efbc8-70fc-4735-92fa-89417acace8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:03.361207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:03.362127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:03.363010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:03.363010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:03.363010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:03.364163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 325/645] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:03.365025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455efbc8-70fc-4735-92fa-89417acace8c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.392941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:03.432621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072aee4-a142-4b41-b1ab-8db9eae15125 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.472619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdc2bb-d853-4e19-8f8c-475c8368c3b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.473273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdc2bb-d853-4e19-8f8c-475c8368c3b2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.473541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdc2bb-d853-4e19-8f8c-475c8368c3b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.473812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdc2bb-d853-4e19-8f8c-475c8368c3b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.474451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdc2bb-d853-4e19-8f8c-475c8368c3b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:03.474740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdc2bb-d853-4e19-8f8c-475c8368c3b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:03.476144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdc2bb-d853-4e19-8f8c-475c8368c3b2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:03.480892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455efbc8-70fc-4735-92fa-89417acace8c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.488254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 321/646] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:03.493405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdc2bb-d853-4e19-8f8c-475c8368c3b2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.510017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:03.516511 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df8f11-9a24-4923-a1c8-fcb49fd84272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.517112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df8f11-9a24-4923-a1c8-fcb49fd84272 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.517685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df8f11-9a24-4923-a1c8-fcb49fd84272 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.517685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df8f11-9a24-4923-a1c8-fcb49fd84272 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.518301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df8f11-9a24-4923-a1c8-fcb49fd84272 None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:03.518634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df8f11-9a24-4923-a1c8-fcb49fd84272 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:03.520088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df8f11-9a24-4923-a1c8-fcb49fd84272 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:03.532940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 326/647] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:01:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:03.532940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df8f11-9a24-4923-a1c8-fcb49fd84272 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.605086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdc2bb-d853-4e19-8f8c-475c8368c3b2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.632757 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:03.643802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf14e40-30b5-4a23-b76b-e469f90a5d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.644425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf14e40-30b5-4a23-b76b-e469f90a5d0d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.645223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf14e40-30b5-4a23-b76b-e469f90a5d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.645540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf14e40-30b5-4a23-b76b-e469f90a5d0d None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.646273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf14e40-30b5-4a23-b76b-e469f90a5d0d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:03.646883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf14e40-30b5-4a23-b76b-e469f90a5d0d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:03.648110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf14e40-30b5-4a23-b76b-e469f90a5d0d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:03.654219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df8f11-9a24-4923-a1c8-fcb49fd84272 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.671540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 322/648] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 10 14:01:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:03.673284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf14e40-30b5-4a23-b76b-e469f90a5d0d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.691567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0e8fa-70e6-4a77-8e8e-844d7174dbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.692184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0e8fa-70e6-4a77-8e8e-844d7174dbcd None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.692464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0e8fa-70e6-4a77-8e8e-844d7174dbcd None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.692738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0e8fa-70e6-4a77-8e8e-844d7174dbcd None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.693418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0e8fa-70e6-4a77-8e8e-844d7174dbcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:03.693726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0e8fa-70e6-4a77-8e8e-844d7174dbcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:03.695272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f0e8fa-70e6-4a77-8e8e-844d7174dbcd None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:03.711506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 327/649] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:03.711506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0e8fa-70e6-4a77-8e8e-844d7174dbcd None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.753848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:03.784350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf14e40-30b5-4a23-b76b-e469f90a5d0d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.810870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0e8fa-70e6-4a77-8e8e-844d7174dbcd None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.818387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93551e0-b609-4578-a3fe-d656e8ba136c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.819012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93551e0-b609-4578-a3fe-d656e8ba136c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.819299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93551e0-b609-4578-a3fe-d656e8ba136c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.819606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93551e0-b609-4578-a3fe-d656e8ba136c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.826235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93551e0-b609-4578-a3fe-d656e8ba136c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:03.827708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93551e0-b609-4578-a3fe-d656e8ba136c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c93551e0-b609-4578-a3fe-d656e8ba136c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:03.839105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:03.840978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:03.843107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:03.843107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:03.843107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:03.843107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:03.843107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:03.843107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:03.843107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:03.843107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:03.843107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 323/650] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:03.843107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93551e0-b609-4578-a3fe-d656e8ba136c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.852220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37c5d1-6ab8-42ca-9041-4c1c8581406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.852811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c5d1-6ab8-42ca-9041-4c1c8581406c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.853087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c5d1-6ab8-42ca-9041-4c1c8581406c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.853360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a37c5d1-6ab8-42ca-9041-4c1c8581406c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.865089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37c5d1-6ab8-42ca-9041-4c1c8581406c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:03.866529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a37c5d1-6ab8-42ca-9041-4c1c8581406c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:03.873152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-8a37c5d1-6ab8-42ca-9041-4c1c8581406c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:03.882679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:03.883865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:03.884761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:03.884761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:03.884761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:03.884761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 328/651] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:03.884994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37c5d1-6ab8-42ca-9041-4c1c8581406c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:03.943368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93551e0-b609-4578-a3fe-d656e8ba136c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:03.971531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:03.977365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810cc79-a755-4d7e-ae54-f133caa892b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:03.977962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810cc79-a755-4d7e-ae54-f133caa892b3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:03.978235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810cc79-a755-4d7e-ae54-f133caa892b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:03.978507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d810cc79-a755-4d7e-ae54-f133caa892b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:03.983728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a37c5d1-6ab8-42ca-9041-4c1c8581406c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:04.017113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6404c21d-85bc-44c5-8cfc-29a799a5e7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:04.017736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404c21d-85bc-44c5-8cfc-29a799a5e7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.018011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404c21d-85bc-44c5-8cfc-29a799a5e7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.018279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6404c21d-85bc-44c5-8cfc-29a799a5e7b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.025541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404c21d-85bc-44c5-8cfc-29a799a5e7b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:04.026981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6404c21d-85bc-44c5-8cfc-29a799a5e7b8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-6404c21d-85bc-44c5-8cfc-29a799a5e7b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:04.038585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:04.039777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 329/652] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:01:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:04.040706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885519a-8d3f-4247-afd5-0bd87c850568 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.041373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885519a-8d3f-4247-afd5-0bd87c850568 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.041373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0885519a-8d3f-4247-afd5-0bd87c850568 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.042470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 330/653] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:04.043148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885519a-8d3f-4247-afd5-0bd87c850568 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:04.048120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810cc79-a755-4d7e-ae54-f133caa892b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:04.049630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d810cc79-a755-4d7e-ae54-f133caa892b3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:04.102895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:04.145453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0885519a-8d3f-4247-afd5-0bd87c850568 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:04.148912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 324/654] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:01:04.149752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810cc79-a755-4d7e-ae54-f133caa892b3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:04.179249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77543b-7acb-4da3-aee0-53ed20fc1fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:04.179878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77543b-7acb-4da3-aee0-53ed20fc1fde None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.180157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77543b-7acb-4da3-aee0-53ed20fc1fde None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.180430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77543b-7acb-4da3-aee0-53ed20fc1fde None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.180877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:04.181147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77543b-7acb-4da3-aee0-53ed20fc1fde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:04.181453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77543b-7acb-4da3-aee0-53ed20fc1fde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:04.182986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e77543b-7acb-4da3-aee0-53ed20fc1fde None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:04.195011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 331/655] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:04.195833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77543b-7acb-4da3-aee0-53ed20fc1fde None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:04.248750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d810cc79-a755-4d7e-ae54-f133caa892b3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:04.282264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b04113-59ef-45b2-8e3a-197c1ca94c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:04.282913 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b04113-59ef-45b2-8e3a-197c1ca94c44 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.283136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b04113-59ef-45b2-8e3a-197c1ca94c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.283407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b04113-59ef-45b2-8e3a-197c1ca94c44 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.284072 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b04113-59ef-45b2-8e3a-197c1ca94c44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:04.284372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b04113-59ef-45b2-8e3a-197c1ca94c44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:04.285862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b04113-59ef-45b2-8e3a-197c1ca94c44 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:04.292964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77543b-7acb-4da3-aee0-53ed20fc1fde None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:04.297388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 325/656] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:04.298576 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb2e78-13bf-4539-89d1-0ece183b5016 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.298859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb2e78-13bf-4539-89d1-0ece183b5016 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.299372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beeb2e78-13bf-4539-89d1-0ece183b5016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.312339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:04.317345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-beeb2e78-13bf-4539-89d1-0ece183b5016 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:04.318632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-beeb2e78-13bf-4539-89d1-0ece183b5016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:04.329848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b367414-d7e6-4126-9ecf-77979c82e73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:04.330446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b367414-d7e6-4126-9ecf-77979c82e73c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.330724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b367414-d7e6-4126-9ecf-77979c82e73c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.330997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b367414-d7e6-4126-9ecf-77979c82e73c None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.331751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b367414-d7e6-4126-9ecf-77979c82e73c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:04.332050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b367414-d7e6-4126-9ecf-77979c82e73c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:04.334134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b367414-d7e6-4126-9ecf-77979c82e73c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:04.351672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 332/657] 10.222.0.26 () {56 vars in 1137 bytes} [Sat Sep 10 14:01:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:04.352633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b367414-d7e6-4126-9ecf-77979c82e73c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:04.385661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 326/658] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:04.386497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb2e78-13bf-4539-89d1-0ece183b5016 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:04.410117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:04.474917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b367414-d7e6-4126-9ecf-77979c82e73c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:04.499663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beeb2e78-13bf-4539-89d1-0ece183b5016 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:04.510446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e7255-2a46-4993-b3fe-677c06e838b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:04.511072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7255-2a46-4993-b3fe-677c06e838b0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.511352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7255-2a46-4993-b3fe-677c06e838b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.511663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1e7255-2a46-4993-b3fe-677c06e838b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.521650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e7255-2a46-4993-b3fe-677c06e838b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:04.523093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1e7255-2a46-4993-b3fe-677c06e838b0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:04.535865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a504f4-ce9d-4fe2-baa3-4a772dc23771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:04.536696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a504f4-ce9d-4fe2-baa3-4a772dc23771 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.537152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a504f4-ce9d-4fe2-baa3-4a772dc23771 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-ff1e7255-2a46-4993-b3fe-677c06e838b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:04.537530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:04.538402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:04.539698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:04.539698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:04.539698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:04.539698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 333/659] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:04.539698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e7255-2a46-4993-b3fe-677c06e838b0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:04.540204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a504f4-ce9d-4fe2-baa3-4a772dc23771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.540935 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a504f4-ce9d-4fe2-baa3-4a772dc23771 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=cb24a1b55ee7408ab24c68991aec54ec) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:04.541242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a504f4-ce9d-4fe2-baa3-4a772dc23771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:04.542672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a504f4-ce9d-4fe2-baa3-4a772dc23771 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:04.551893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 327/660] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:04 2022] GET /identity/v3/role_assignments?user.id=cb24a1b55ee7408ab24c68991aec54ec&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:04.553036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a504f4-ce9d-4fe2-baa3-4a772dc23771 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:04.579896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:04.665592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:04.677797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1e7255-2a46-4993-b3fe-677c06e838b0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:04.682008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a504f4-ce9d-4fe2-baa3-4a772dc23771 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:04.717136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b512992-b124-42c8-a1b0-c5d9ba183f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:04.717780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b512992-b124-42c8-a1b0-c5d9ba183f91 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.718056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b512992-b124-42c8-a1b0-c5d9ba183f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.718320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b512992-b124-42c8-a1b0-c5d9ba183f91 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.720303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f538f7a-4133-4aaf-b15d-1bc499d8efba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:04.720915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f538f7a-4133-4aaf-b15d-1bc499d8efba None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.721187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f538f7a-4133-4aaf-b15d-1bc499d8efba None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.721454 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f538f7a-4133-4aaf-b15d-1bc499d8efba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.727304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b512992-b124-42c8-a1b0-c5d9ba183f91 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:04.728792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b512992-b124-42c8-a1b0-c5d9ba183f91 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-0b512992-b124-42c8-a1b0-c5d9ba183f91 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:04.741651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:04.742816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:04.743786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 334/661] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:01:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:04.745167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b512992-b124-42c8-a1b0-c5d9ba183f91 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:04.806858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f538f7a-4133-4aaf-b15d-1bc499d8efba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:04.808900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f538f7a-4133-4aaf-b15d-1bc499d8efba None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:04.842323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:04.856515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b512992-b124-42c8-a1b0-c5d9ba183f91 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:04.893096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b555e8-0a73-49bd-92b6-b47ac26053c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:04.893700 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b555e8-0a73-49bd-92b6-b47ac26053c6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:04.893976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b555e8-0a73-49bd-92b6-b47ac26053c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:04.894247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b555e8-0a73-49bd-92b6-b47ac26053c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:04.895437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b555e8-0a73-49bd-92b6-b47ac26053c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:04.895758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b555e8-0a73-49bd-92b6-b47ac26053c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:04.897125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b555e8-0a73-49bd-92b6-b47ac26053c6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:04.901430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:04.908978 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 335/662] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:04.913578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b555e8-0a73-49bd-92b6-b47ac26053c6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:04.927654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 328/663] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:01:04.928930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f538f7a-4133-4aaf-b15d-1bc499d8efba None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.014737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b555e8-0a73-49bd-92b6-b47ac26053c6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.036334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f538f7a-4133-4aaf-b15d-1bc499d8efba None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.049815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f6460-e56a-4eaf-bc8a-d887a6b1e87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.050713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f6460-e56a-4eaf-bc8a-d887a6b1e87c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.050713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f6460-e56a-4eaf-bc8a-d887a6b1e87c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.051127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333f6460-e56a-4eaf-bc8a-d887a6b1e87c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.061903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333f6460-e56a-4eaf-bc8a-d887a6b1e87c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-333f6460-e56a-4eaf-bc8a-d887a6b1e87c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:05.067357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:05.068584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:05.069399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 336/664] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:01:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:05.070693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f6460-e56a-4eaf-bc8a-d887a6b1e87c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.078723 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da605e2d-4a56-401a-b807-b3d077c27662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.080165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da605e2d-4a56-401a-b807-b3d077c27662 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.080625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da605e2d-4a56-401a-b807-b3d077c27662 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.081482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:05.082453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da605e2d-4a56-401a-b807-b3d077c27662 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.084546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da605e2d-4a56-401a-b807-b3d077c27662 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:05.084944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da605e2d-4a56-401a-b807-b3d077c27662 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:05.086557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da605e2d-4a56-401a-b807-b3d077c27662 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:05.108654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 329/665] 10.222.0.26 () {56 vars in 1139 bytes} [Sat Sep 10 14:01:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:05.109837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da605e2d-4a56-401a-b807-b3d077c27662 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.158034 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:05.179136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333f6460-e56a-4eaf-bc8a-d887a6b1e87c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.214680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858df48-7976-4835-b92e-cdd927af4033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.215301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858df48-7976-4835-b92e-cdd927af4033 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.215607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858df48-7976-4835-b92e-cdd927af4033 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.215875 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858df48-7976-4835-b92e-cdd927af4033 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.215975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da605e2d-4a56-401a-b807-b3d077c27662 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.223349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858df48-7976-4835-b92e-cdd927af4033 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:05.224988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858df48-7976-4835-b92e-cdd927af4033 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-a858df48-7976-4835-b92e-cdd927af4033 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:05.236317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:05.237376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:05.238813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:05.238813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:05.238813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:05.238813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:05.238813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:05.238813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:05.238813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:05.238813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:05.238813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 337/666] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:05.238813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858df48-7976-4835-b92e-cdd927af4033 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.256323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cf0ab-eb59-46d5-8a9e-e494d069d037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.256946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cf0ab-eb59-46d5-8a9e-e494d069d037 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.257218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cf0ab-eb59-46d5-8a9e-e494d069d037 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.257491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0cf0ab-eb59-46d5-8a9e-e494d069d037 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.267502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cf0ab-eb59-46d5-8a9e-e494d069d037 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:05.268972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0cf0ab-eb59-46d5-8a9e-e494d069d037 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-ee0cf0ab-eb59-46d5-8a9e-e494d069d037 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:05.285086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:05.286544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:05.287426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:05.287426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:05.287426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:05.290885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 330/667] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:05.291757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cf0ab-eb59-46d5-8a9e-e494d069d037 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.308761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:05.346282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858df48-7976-4835-b92e-cdd927af4033 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.385124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508394b4-f849-417d-bb54-87898373d546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.385734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508394b4-f849-417d-bb54-87898373d546 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.386010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508394b4-f849-417d-bb54-87898373d546 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.386278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508394b4-f849-417d-bb54-87898373d546 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.386932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508394b4-f849-417d-bb54-87898373d546 None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:05.387229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508394b4-f849-417d-bb54-87898373d546 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:05.388638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508394b4-f849-417d-bb54-87898373d546 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:05.390199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:05.401982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 338/668] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:01:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:05.402152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0cf0ab-eb59-46d5-8a9e-e494d069d037 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.402233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508394b4-f849-417d-bb54-87898373d546 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.447722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57830f-9cf8-4ee4-be81-5b4aafd24f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.448328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57830f-9cf8-4ee4-be81-5b4aafd24f33 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.448604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57830f-9cf8-4ee4-be81-5b4aafd24f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.448870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57830f-9cf8-4ee4-be81-5b4aafd24f33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.512892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508394b4-f849-417d-bb54-87898373d546 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.517828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57830f-9cf8-4ee4-be81-5b4aafd24f33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:05.519311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57830f-9cf8-4ee4-be81-5b4aafd24f33 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:05.545214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:05.552157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f6dff-6377-495e-89d9-f5c0e0afd155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.552789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f6dff-6377-495e-89d9-f5c0e0afd155 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.553064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f6dff-6377-495e-89d9-f5c0e0afd155 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.553330 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f6dff-6377-495e-89d9-f5c0e0afd155 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.554133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f6dff-6377-495e-89d9-f5c0e0afd155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:05.554546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f6dff-6377-495e-89d9-f5c0e0afd155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:05.556023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f6dff-6377-495e-89d9-f5c0e0afd155 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:05.574847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 339/669] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:05.574847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f6dff-6377-495e-89d9-f5c0e0afd155 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.627297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:05.639537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 331/670] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:01:05.639537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57830f-9cf8-4ee4-be81-5b4aafd24f33 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.680246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f6dff-6377-495e-89d9-f5c0e0afd155 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.721609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c3171-4544-42a8-a145-8de0a2cc6c69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.722210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c3171-4544-42a8-a145-8de0a2cc6c69 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.722486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c3171-4544-42a8-a145-8de0a2cc6c69 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.723069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c3171-4544-42a8-a145-8de0a2cc6c69 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.724165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c3171-4544-42a8-a145-8de0a2cc6c69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:05.724469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c3171-4544-42a8-a145-8de0a2cc6c69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:05.726227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c3171-4544-42a8-a145-8de0a2cc6c69 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:05.743203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 340/671] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:05.743203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c3171-4544-42a8-a145-8de0a2cc6c69 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.743487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57830f-9cf8-4ee4-be81-5b4aafd24f33 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.779142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c990a-0c6a-4e1c-bf28-8ddd6726553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.779812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c990a-0c6a-4e1c-bf28-8ddd6726553f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.780089 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c990a-0c6a-4e1c-bf28-8ddd6726553f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.780360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c990a-0c6a-4e1c-bf28-8ddd6726553f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.790660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c990a-0c6a-4e1c-bf28-8ddd6726553f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:05.792145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c990a-0c6a-4e1c-bf28-8ddd6726553f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:05.797467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-bb5c990a-0c6a-4e1c-bf28-8ddd6726553f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:05.808943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:05.809921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:05.810825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:05.810825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:05.810825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:05.810825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 332/672] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:05.810825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c990a-0c6a-4e1c-bf28-8ddd6726553f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.855219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c3171-4544-42a8-a145-8de0a2cc6c69 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.865279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:05.892545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e95cd-8d61-4267-a0ea-57b12a1052ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.893075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e95cd-8d61-4267-a0ea-57b12a1052ce None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.893356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e95cd-8d61-4267-a0ea-57b12a1052ce None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.893630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e95cd-8d61-4267-a0ea-57b12a1052ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.894362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e95cd-8d61-4267-a0ea-57b12a1052ce None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=cf75b583add44b6b87bff5dc67080d1c) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:05.894808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e95cd-8d61-4267-a0ea-57b12a1052ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:05.896322 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e95cd-8d61-4267-a0ea-57b12a1052ce None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:05.902346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 341/673] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:05 2022] GET /identity/v3/role_assignments?user.id=cf75b583add44b6b87bff5dc67080d1c&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:05.902921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e95cd-8d61-4267-a0ea-57b12a1052ce None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:05.931057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c990a-0c6a-4e1c-bf28-8ddd6726553f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:05.972159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b360fb-6486-4e52-b767-78594f67bd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:05.973815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b360fb-6486-4e52-b767-78594f67bd88 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:05.974431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b360fb-6486-4e52-b767-78594f67bd88 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:05.974765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b360fb-6486-4e52-b767-78594f67bd88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:05.990012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b360fb-6486-4e52-b767-78594f67bd88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:05.991582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b360fb-6486-4e52-b767-78594f67bd88 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-07b360fb-6486-4e52-b767-78594f67bd88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:06.017105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:06.018136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:06.019032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:06.019032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:06.019032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:06.027397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 333/674] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:06.027397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b360fb-6486-4e52-b767-78594f67bd88 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:06.039235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e95cd-8d61-4267-a0ea-57b12a1052ce None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:06.055823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:06.079917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8db297-4262-4c58-b65f-7256a2869024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:06.080608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8db297-4262-4c58-b65f-7256a2869024 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:06.080889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8db297-4262-4c58-b65f-7256a2869024 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:06.081160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8db297-4262-4c58-b65f-7256a2869024 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:06.089583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8db297-4262-4c58-b65f-7256a2869024 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:06.091118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8db297-4262-4c58-b65f-7256a2869024 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-1f8db297-4262-4c58-b65f-7256a2869024 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:06.109154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:06.110081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:06.110963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:06.110963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:06.110963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:06.110963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:06.110963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:06.110963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:06.110963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:06.110963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:06.114181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 342/675] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:06.115991 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8db297-4262-4c58-b65f-7256a2869024 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:06.162089 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:06.165215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b360fb-6486-4e52-b767-78594f67bd88 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:06.207999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd70ff-2445-461b-817c-42146d5b7a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:06.208820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd70ff-2445-461b-817c-42146d5b7a79 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:06.209599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd70ff-2445-461b-817c-42146d5b7a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:06.209599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bd70ff-2445-461b-817c-42146d5b7a79 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:06.210017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd70ff-2445-461b-817c-42146d5b7a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:06.210318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd70ff-2445-461b-817c-42146d5b7a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:06.211726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bd70ff-2445-461b-817c-42146d5b7a79 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:06.228110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 334/676] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:06.232910 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd70ff-2445-461b-817c-42146d5b7a79 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:06.249427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8db297-4262-4c58-b65f-7256a2869024 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:06.293897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3a21c-9892-4608-a05b-46d570613f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:06.294778 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3a21c-9892-4608-a05b-46d570613f35 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:06.295061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3a21c-9892-4608-a05b-46d570613f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:06.295814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3a21c-9892-4608-a05b-46d570613f35 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:06.296754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3a21c-9892-4608-a05b-46d570613f35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:06.297067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3a21c-9892-4608-a05b-46d570613f35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:06.299162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3a21c-9892-4608-a05b-46d570613f35 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:06.319206 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 343/677] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:06.320117 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3a21c-9892-4608-a05b-46d570613f35 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:06.350823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bd70ff-2445-461b-817c-42146d5b7a79 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:06.364801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:06.388535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28540b75-a138-473f-9055-7497cbcad6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:06.389294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28540b75-a138-473f-9055-7497cbcad6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:06.389758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28540b75-a138-473f-9055-7497cbcad6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:06.390129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28540b75-a138-473f-9055-7497cbcad6a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:06.390972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28540b75-a138-473f-9055-7497cbcad6a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:06.391380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28540b75-a138-473f-9055-7497cbcad6a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:06.392937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28540b75-a138-473f-9055-7497cbcad6a4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:06.406061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 335/678] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:06.407170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28540b75-a138-473f-9055-7497cbcad6a4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:06.443906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:06.452858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3a21c-9892-4608-a05b-46d570613f35 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:06.494058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba15e9c-2a45-4596-86d2-4bc914979b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:06.495125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba15e9c-2a45-4596-86d2-4bc914979b80 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:06.495670 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba15e9c-2a45-4596-86d2-4bc914979b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:06.496176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba15e9c-2a45-4596-86d2-4bc914979b80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:06.504521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba15e9c-2a45-4596-86d2-4bc914979b80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:06.506272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba15e9c-2a45-4596-86d2-4bc914979b80 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-1ba15e9c-2a45-4596-86d2-4bc914979b80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:06.518870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:06.520523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:06.521466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:06.521466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:06.521466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:06.521466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:06.521466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:06.521466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:06.521466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:06.521466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:06.521466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:01:06.521466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:06.524040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 344/679] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:06.525139 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba15e9c-2a45-4596-86d2-4bc914979b80 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:06.539021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28540b75-a138-473f-9055-7497cbcad6a4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:06.581475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b833c-8f0c-44ae-a8fe-5cb56e93893f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:06.582959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b833c-8f0c-44ae-a8fe-5cb56e93893f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:06.583245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b833c-8f0c-44ae-a8fe-5cb56e93893f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:06.583561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b833c-8f0c-44ae-a8fe-5cb56e93893f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:06.591088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b833c-8f0c-44ae-a8fe-5cb56e93893f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:06.592792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b833c-8f0c-44ae-a8fe-5cb56e93893f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-cb6b833c-8f0c-44ae-a8fe-5cb56e93893f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:06.605029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:06.605971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:06.606923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:06.606923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:06.606923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:06.606923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:06.606923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:06.606923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:06.606923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:06.606923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:06.610871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 336/680] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:06.611758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b833c-8f0c-44ae-a8fe-5cb56e93893f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:06.634202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:06.643684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba15e9c-2a45-4596-86d2-4bc914979b80 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:06.692436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81d2f8-8170-4dca-b850-e575a16b2c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:06.693391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81d2f8-8170-4dca-b850-e575a16b2c01 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:06.694071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81d2f8-8170-4dca-b850-e575a16b2c01 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:06.694550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d81d2f8-8170-4dca-b850-e575a16b2c01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:06.696085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81d2f8-8170-4dca-b850-e575a16b2c01 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=fdc278f50e794e12a27d30fca9d131c1) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:06.696623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81d2f8-8170-4dca-b850-e575a16b2c01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:06.699183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d81d2f8-8170-4dca-b850-e575a16b2c01 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:06.701370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:06.709822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 345/681] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:06 2022] GET /identity/v3/role_assignments?user.id=fdc278f50e794e12a27d30fca9d131c1&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:06.711514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81d2f8-8170-4dca-b850-e575a16b2c01 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:06.724785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b833c-8f0c-44ae-a8fe-5cb56e93893f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:06.804810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc004e-6f35-4a82-aa62-392914d46b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:06.805493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc004e-6f35-4a82-aa62-392914d46b4b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:06.805776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc004e-6f35-4a82-aa62-392914d46b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:06.806047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc004e-6f35-4a82-aa62-392914d46b4b None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:06.806802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc004e-6f35-4a82-aa62-392914d46b4b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:06.807105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc004e-6f35-4a82-aa62-392914d46b4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:06.810387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc004e-6f35-4a82-aa62-392914d46b4b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:06.846703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 337/682] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 10 14:01:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:06.858080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc004e-6f35-4a82-aa62-392914d46b4b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:06.904658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d81d2f8-8170-4dca-b850-e575a16b2c01 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:06.949416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1bcf91-9f88-444c-982e-5b1ee32bb15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:06.950113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bcf91-9f88-444c-982e-5b1ee32bb15f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:06.950390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bcf91-9f88-444c-982e-5b1ee32bb15f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:06.951175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1bcf91-9f88-444c-982e-5b1ee32bb15f None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:06.951950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bcf91-9f88-444c-982e-5b1ee32bb15f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:06.952669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bcf91-9f88-444c-982e-5b1ee32bb15f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:06.954283 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1bcf91-9f88-444c-982e-5b1ee32bb15f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:06.972786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 346/683] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:06.974673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c2cd7-9eca-4159-8715-a009793f9778 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:06.974959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c2cd7-9eca-4159-8715-a009793f9778 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:06.975229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783c2cd7-9eca-4159-8715-a009793f9778 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:06.978650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 347/684] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:06.979409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c2cd7-9eca-4159-8715-a009793f9778 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:06.996894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc004e-6f35-4a82-aa62-392914d46b4b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:07.001711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:07.034950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d28d9-0240-4cee-bce3-d6fa02e6497c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:07.036192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d28d9-0240-4cee-bce3-d6fa02e6497c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:07.036481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d28d9-0240-4cee-bce3-d6fa02e6497c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:07.036736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7d28d9-0240-4cee-bce3-d6fa02e6497c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:07.063857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:07.112617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d28d9-0240-4cee-bce3-d6fa02e6497c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:07.114188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7d28d9-0240-4cee-bce3-d6fa02e6497c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:07.120362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783c2cd7-9eca-4159-8715-a009793f9778 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:07.165304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7426c4-2648-46d8-a23f-d5047c39ed46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:07.166415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7426c4-2648-46d8-a23f-d5047c39ed46 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:07.167178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7426c4-2648-46d8-a23f-d5047c39ed46 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:07.167680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7426c4-2648-46d8-a23f-d5047c39ed46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:07.168649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7426c4-2648-46d8-a23f-d5047c39ed46 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=12c71caf31354ec193e3e2da231f1e77) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:07.169172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7426c4-2648-46d8-a23f-d5047c39ed46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:07.171542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7426c4-2648-46d8-a23f-d5047c39ed46 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:07.180520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 348/685] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:06 2022] GET /identity/v3/role_assignments?user.id=12c71caf31354ec193e3e2da231f1e77&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:07.182078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da8535-2c4d-49bf-b0e2-2bc717cce65e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:07.182593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da8535-2c4d-49bf-b0e2-2bc717cce65e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:07.183060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20da8535-2c4d-49bf-b0e2-2bc717cce65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:07.221096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-20da8535-2c4d-49bf-b0e2-2bc717cce65e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:07.223233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-20da8535-2c4d-49bf-b0e2-2bc717cce65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:07.276949 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 338/686] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:01:07.282172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d28d9-0240-4cee-bce3-d6fa02e6497c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:07.351279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:07.387276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 349/687] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:07.465994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7d28d9-0240-4cee-bce3-d6fa02e6497c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:07.513873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e78008-4c45-4ed9-ac17-e2b1e4359f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:07.514432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e78008-4c45-4ed9-ac17-e2b1e4359f12 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:07.514711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e78008-4c45-4ed9-ac17-e2b1e4359f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:07.514984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e78008-4c45-4ed9-ac17-e2b1e4359f12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:07.526445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e78008-4c45-4ed9-ac17-e2b1e4359f12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:07.528496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e78008-4c45-4ed9-ac17-e2b1e4359f12 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-76e78008-4c45-4ed9-ac17-e2b1e4359f12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:07.553102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:07.554127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:07.555001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:07.555001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:07.555001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:07.555001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 339/688] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:07.560605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da8535-2c4d-49bf-b0e2-2bc717cce65e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:07.590239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:07.702285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20da8535-2c4d-49bf-b0e2-2bc717cce65e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:07.732431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e78008-4c45-4ed9-ac17-e2b1e4359f12 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:07.761977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468947c-88b5-47c6-8501-0f7ef211a975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:07.762595 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468947c-88b5-47c6-8501-0f7ef211a975 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:07.762869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468947c-88b5-47c6-8501-0f7ef211a975 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:07.763132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468947c-88b5-47c6-8501-0f7ef211a975 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:07.764023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468947c-88b5-47c6-8501-0f7ef211a975 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:07.764264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468947c-88b5-47c6-8501-0f7ef211a975 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:07.766323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e468947c-88b5-47c6-8501-0f7ef211a975 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:07.786550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 350/689] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:07.792132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468947c-88b5-47c6-8501-0f7ef211a975 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:07.870049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:07.886027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e78008-4c45-4ed9-ac17-e2b1e4359f12 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:07.931040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:07.958658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfe208-6562-4be5-820e-0883979f3472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:07.959296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe208-6562-4be5-820e-0883979f3472 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:07.959602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe208-6562-4be5-820e-0883979f3472 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:07.959921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfe208-6562-4be5-820e-0883979f3472 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:08.004628 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfe208-6562-4be5-820e-0883979f3472 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-26bfe208-6562-4be5-820e-0883979f3472 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:08.010196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:08.011143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:08.012088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:08.023535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 340/690] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:08.026420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468947c-88b5-47c6-8501-0f7ef211a975 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:08.035707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfe208-6562-4be5-820e-0883979f3472 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:08.120383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fe311-57b2-43a1-97dc-5bcfd428266d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:08.121037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fe311-57b2-43a1-97dc-5bcfd428266d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:08.121306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fe311-57b2-43a1-97dc-5bcfd428266d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:08.123235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fe311-57b2-43a1-97dc-5bcfd428266d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:08.141220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fe311-57b2-43a1-97dc-5bcfd428266d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:08.145688 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1fe311-57b2-43a1-97dc-5bcfd428266d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-4c1fe311-57b2-43a1-97dc-5bcfd428266d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:08.173120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:08.174053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:08.175002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:08.175002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:08.175002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:08.175002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:08.175002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:08.175002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:08.175002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:08.175002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:08.179462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 351/691] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 386 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:08.180607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fe311-57b2-43a1-97dc-5bcfd428266d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:08.401782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fe311-57b2-43a1-97dc-5bcfd428266d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:08.458923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfe208-6562-4be5-820e-0883979f3472 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:08.487070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:08.495932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549671ae-aa90-4a8d-b4c4-b5c6f0665b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:08.496562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549671ae-aa90-4a8d-b4c4-b5c6f0665b93 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:08.496846 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549671ae-aa90-4a8d-b4c4-b5c6f0665b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:08.497123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549671ae-aa90-4a8d-b4c4-b5c6f0665b93 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:08.497972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549671ae-aa90-4a8d-b4c4-b5c6f0665b93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:08.498488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549671ae-aa90-4a8d-b4c4-b5c6f0665b93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:08.500306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549671ae-aa90-4a8d-b4c4-b5c6f0665b93 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:08.536191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 352/692] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:08.544462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549671ae-aa90-4a8d-b4c4-b5c6f0665b93 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:08.558339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32d05a1-a08f-4042-b3e8-a312777135ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:08.559129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d05a1-a08f-4042-b3e8-a312777135ce None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:08.559548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d05a1-a08f-4042-b3e8-a312777135ce None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:08.559922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32d05a1-a08f-4042-b3e8-a312777135ce None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:08.560664 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d05a1-a08f-4042-b3e8-a312777135ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:08.561111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d05a1-a08f-4042-b3e8-a312777135ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:08.562661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32d05a1-a08f-4042-b3e8-a312777135ce None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:08.572197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 341/693] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:01:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 544 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:08.573542 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e576222-b990-47e4-8dff-a5ef339e5239 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:08.573897 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e576222-b990-47e4-8dff-a5ef339e5239 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:08.574237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e576222-b990-47e4-8dff-a5ef339e5239 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:08.575970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 342/694] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:08.581481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e576222-b990-47e4-8dff-a5ef339e5239 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:08.600496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:08.655603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549671ae-aa90-4a8d-b4c4-b5c6f0665b93 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:08.699469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b99cf-b17f-4752-bcab-63212feff30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:08.699469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b99cf-b17f-4752-bcab-63212feff30f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:08.699469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b99cf-b17f-4752-bcab-63212feff30f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:08.699469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b99cf-b17f-4752-bcab-63212feff30f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:08.699957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b99cf-b17f-4752-bcab-63212feff30f None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=bc0a7e1b772b4703ac2e5ec33b5ecf6b) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:08.699957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b99cf-b17f-4752-bcab-63212feff30f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:08.699957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b99cf-b17f-4752-bcab-63212feff30f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:08.702212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e576222-b990-47e4-8dff-a5ef339e5239 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:08.707350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 353/695] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:08 2022] GET /identity/v3/role_assignments?user.id=bc0a7e1b772b4703ac2e5ec33b5ecf6b&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:08.708725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f32a14-e0db-459a-a5b4-cd1de5c97128 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:08.709015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f32a14-e0db-459a-a5b4-cd1de5c97128 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:08.709292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f32a14-e0db-459a-a5b4-cd1de5c97128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:08.731147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e9f32a14-e0db-459a-a5b4-cd1de5c97128 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:08.732493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e9f32a14-e0db-459a-a5b4-cd1de5c97128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:08.772824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27349214-1be8-4b7b-960d-3b968ac3d84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:08.773487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27349214-1be8-4b7b-960d-3b968ac3d84f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:08.773760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27349214-1be8-4b7b-960d-3b968ac3d84f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:08.774028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27349214-1be8-4b7b-960d-3b968ac3d84f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:08.805154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27349214-1be8-4b7b-960d-3b968ac3d84f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:08.807008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27349214-1be8-4b7b-960d-3b968ac3d84f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-27349214-1be8-4b7b-960d-3b968ac3d84f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:08.843526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:08.844535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:08.846294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:08.846294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:08.846294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:08.846294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 343/696] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:08.850346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:08.851956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27349214-1be8-4b7b-960d-3b968ac3d84f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:08.892867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 354/697] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:08.939964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:08.982363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27349214-1be8-4b7b-960d-3b968ac3d84f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:09.018437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482c630-58cb-4713-b0a4-47c747353c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:09.019050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482c630-58cb-4713-b0a4-47c747353c42 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:09.019325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482c630-58cb-4713-b0a4-47c747353c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:09.020174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482c630-58cb-4713-b0a4-47c747353c42 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:09.020987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482c630-58cb-4713-b0a4-47c747353c42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:09.021283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482c630-58cb-4713-b0a4-47c747353c42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:09.022734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482c630-58cb-4713-b0a4-47c747353c42 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:09.036017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 344/698] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:09.041678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f32a14-e0db-459a-a5b4-cd1de5c97128 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:09.149822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f32a14-e0db-459a-a5b4-cd1de5c97128 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:09.184861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b150095-7c78-4e2b-92c6-ab560ce43e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:09.185815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b150095-7c78-4e2b-92c6-ab560ce43e5d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:09.186088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b150095-7c78-4e2b-92c6-ab560ce43e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:09.186352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b150095-7c78-4e2b-92c6-ab560ce43e5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:09.195540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b150095-7c78-4e2b-92c6-ab560ce43e5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:09.195805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b150095-7c78-4e2b-92c6-ab560ce43e5d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-4b150095-7c78-4e2b-92c6-ab560ce43e5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:09.210615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:09.211608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:09.212565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:09.212565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:09.212565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:09.212565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:09.212565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:09.212565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:09.212565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:09.212565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:09.212565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:01:09.212565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:09.213089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 355/699] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 10 14:01:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:09.215390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecfa89-4a86-4cce-aa8b-76f2bb622819 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:09.215706 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecfa89-4a86-4cce-aa8b-76f2bb622819 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:09.215982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecfa89-4a86-4cce-aa8b-76f2bb622819 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:09.217431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 345/700] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:09.218603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b150095-7c78-4e2b-92c6-ab560ce43e5d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:09.224920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69780ad6-d78a-4277-930e-4edcf01ed5c3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:09.225205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69780ad6-d78a-4277-930e-4edcf01ed5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:09.225480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69780ad6-d78a-4277-930e-4edcf01ed5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:09.260544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-69780ad6-d78a-4277-930e-4edcf01ed5c3 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:09.261769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-69780ad6-d78a-4277-930e-4edcf01ed5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:09.270034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:09.351268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b150095-7c78-4e2b-92c6-ab560ce43e5d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:09.351586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 346/701] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:09.352667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69780ad6-d78a-4277-930e-4edcf01ed5c3 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:09.389276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44815c-87ef-4a94-9066-0bfcff433cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:09.389276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44815c-87ef-4a94-9066-0bfcff433cec None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:09.389276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44815c-87ef-4a94-9066-0bfcff433cec None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:09.389276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44815c-87ef-4a94-9066-0bfcff433cec None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:09.389717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44815c-87ef-4a94-9066-0bfcff433cec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:09.389717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44815c-87ef-4a94-9066-0bfcff433cec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:09.389717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44815c-87ef-4a94-9066-0bfcff433cec None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:09.407428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 356/702] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 10 14:01:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:09.413605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44815c-87ef-4a94-9066-0bfcff433cec None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:09.437830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:09.483767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69780ad6-d78a-4277-930e-4edcf01ed5c3 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:09.522433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65baa30c-ebac-4c00-90d0-f58fb0697dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:09.523559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baa30c-ebac-4c00-90d0-f58fb0697dff None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:09.524066 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baa30c-ebac-4c00-90d0-f58fb0697dff None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:09.524555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65baa30c-ebac-4c00-90d0-f58fb0697dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:09.537032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65baa30c-ebac-4c00-90d0-f58fb0697dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:09.538756 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65baa30c-ebac-4c00-90d0-f58fb0697dff None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:09.546478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44815c-87ef-4a94-9066-0bfcff433cec None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:09.556435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-65baa30c-ebac-4c00-90d0-f58fb0697dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:09.565659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:09.567319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:09.568426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:09.568426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:09.568426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:09.575335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 347/703] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:09.576498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65baa30c-ebac-4c00-90d0-f58fb0697dff None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:09.583437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4a164-4b60-460c-b9e3-17eb9b609a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:09.584074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4a164-4b60-460c-b9e3-17eb9b609a11 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:09.584351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4a164-4b60-460c-b9e3-17eb9b609a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:09.584624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c4a164-4b60-460c-b9e3-17eb9b609a11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:09.658632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4a164-4b60-460c-b9e3-17eb9b609a11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:09.660284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c4a164-4b60-460c-b9e3-17eb9b609a11 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:09.703134 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65baa30c-ebac-4c00-90d0-f58fb0697dff None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:09.712767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:09.744402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b3224-a1e9-45ba-89c2-8f3c05082e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:09.745030 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b3224-a1e9-45ba-89c2-8f3c05082e80 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:09.745402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b3224-a1e9-45ba-89c2-8f3c05082e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:09.745764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b3224-a1e9-45ba-89c2-8f3c05082e80 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:09.746602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b3224-a1e9-45ba-89c2-8f3c05082e80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:09.746995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b3224-a1e9-45ba-89c2-8f3c05082e80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:09.748478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b3224-a1e9-45ba-89c2-8f3c05082e80 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:09.762113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 348/704] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:09.768241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b3224-a1e9-45ba-89c2-8f3c05082e80 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:09.772383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 357/705] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:01:09.777968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4a164-4b60-460c-b9e3-17eb9b609a11 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:09.781300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:09.881248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b3224-a1e9-45ba-89c2-8f3c05082e80 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:09.897472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c4a164-4b60-460c-b9e3-17eb9b609a11 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:09.922817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902f343-2351-430a-b58a-e8951e42c269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:09.923496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902f343-2351-430a-b58a-e8951e42c269 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:09.923781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902f343-2351-430a-b58a-e8951e42c269 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:09.924054 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7902f343-2351-430a-b58a-e8951e42c269 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:09.924718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902f343-2351-430a-b58a-e8951e42c269 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:09.926772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7902f343-2351-430a-b58a-e8951e42c269 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:09.947646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9d385a-00f2-44e9-a802-75e9b57e3096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:09.948712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d385a-00f2-44e9-a802-75e9b57e3096 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:09.949228 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d385a-00f2-44e9-a802-75e9b57e3096 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:09.949734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9d385a-00f2-44e9-a802-75e9b57e3096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:09.955183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 349/706] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:01:09 2022] POST /identity/v3/users => generated 254 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:09.955183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902f343-2351-430a-b58a-e8951e42c269 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:09.988582 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d385a-00f2-44e9-a802-75e9b57e3096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:09.990620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9d385a-00f2-44e9-a802-75e9b57e3096 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:10.005906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-ba9d385a-00f2-44e9-a802-75e9b57e3096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:10.026606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:10.027675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:10.028561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:10.028561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:10.028561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:10.033140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 358/707] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:10.034462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb38c0-1f41-45b3-8761-8a25d064ef82 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:10.034743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb38c0-1f41-45b3-8761-8a25d064ef82 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:10.035015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb38c0-1f41-45b3-8761-8a25d064ef82 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:10.041407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 359/708] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:10.042315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb38c0-1f41-45b3-8761-8a25d064ef82 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:10.117769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7902f343-2351-430a-b58a-e8951e42c269 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:10.122531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:10.161878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f41c558-fd73-4975-ac93-ce1a84064e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:10.162939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41c558-fd73-4975-ac93-ce1a84064e84 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:10.163414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41c558-fd73-4975-ac93-ce1a84064e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:10.164639 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f41c558-fd73-4975-ac93-ce1a84064e84 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:10.165453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41c558-fd73-4975-ac93-ce1a84064e84 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:10.167043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f41c558-fd73-4975-ac93-ce1a84064e84 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-0f41c558-fd73-4975-ac93-ce1a84064e84 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:10.179595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:10.180627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 14:01:10.181488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:10.183952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 350/709] 10.222.0.26 () {56 vars in 1129 bytes} [Sat Sep 10 14:01:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:10.184793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb38c0-1f41-45b3-8761-8a25d064ef82 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:10.185679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bce2ec-d251-4527-821f-e2cbc71faaaa None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:10.186170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bce2ec-d251-4527-821f-e2cbc71faaaa None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:10.186632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bce2ec-d251-4527-821f-e2cbc71faaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:10.205462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e5bce2ec-d251-4527-821f-e2cbc71faaaa None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:10.207745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e5bce2ec-d251-4527-821f-e2cbc71faaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:10.227590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7e133-5af2-486e-b0ea-a802cd8eadf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:10.228143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7e133-5af2-486e-b0ea-a802cd8eadf6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:10.228424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7e133-5af2-486e-b0ea-a802cd8eadf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:10.228695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d7e133-5af2-486e-b0ea-a802cd8eadf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:10.229613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7e133-5af2-486e-b0ea-a802cd8eadf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:10.229931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7e133-5af2-486e-b0ea-a802cd8eadf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:10.233274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d7e133-5af2-486e-b0ea-a802cd8eadf6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:10.247723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 360/710] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:10.248968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7e133-5af2-486e-b0ea-a802cd8eadf6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:10.292742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 351/711] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:10.293985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bce2ec-d251-4527-821f-e2cbc71faaaa None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:10.312877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:10.418539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d7e133-5af2-486e-b0ea-a802cd8eadf6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:10.449708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:10.451294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bce2ec-d251-4527-821f-e2cbc71faaaa None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:10.485877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e8320-25ce-4bd6-8534-f9a5013850be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:10.486648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e8320-25ce-4bd6-8534-f9a5013850be None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:10.487022 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e8320-25ce-4bd6-8534-f9a5013850be None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:10.487382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e8320-25ce-4bd6-8534-f9a5013850be None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:10.488131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e8320-25ce-4bd6-8534-f9a5013850be None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:10.489078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e8320-25ce-4bd6-8534-f9a5013850be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:10.490563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e8320-25ce-4bd6-8534-f9a5013850be None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:10.501597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 361/712] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 10 14:01:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:10.505934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e8320-25ce-4bd6-8534-f9a5013850be None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:10.516506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c5d4f-f51b-4bc7-87c8-02b34e0aede3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:10.518251 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c5d4f-f51b-4bc7-87c8-02b34e0aede3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:10.518625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c5d4f-f51b-4bc7-87c8-02b34e0aede3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:10.518989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01c5d4f-f51b-4bc7-87c8-02b34e0aede3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:10.531029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c5d4f-f51b-4bc7-87c8-02b34e0aede3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:10.534754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01c5d4f-f51b-4bc7-87c8-02b34e0aede3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a01c5d4f-f51b-4bc7-87c8-02b34e0aede3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:10.554627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:10.556600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:10.558783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:10.558783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:10.558783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:10.558783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:10.558783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:10.558783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:10.558783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:10.558783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:10.563479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 352/713] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:10.572437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c5d4f-f51b-4bc7-87c8-02b34e0aede3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:10.669653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e8320-25ce-4bd6-8534-f9a5013850be None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:10.694611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:10.717549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2c7b6-7ccf-48ac-91c8-c7bda5abb97e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:10.718383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2c7b6-7ccf-48ac-91c8-c7bda5abb97e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:10.718797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2c7b6-7ccf-48ac-91c8-c7bda5abb97e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:10.719123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b2c7b6-7ccf-48ac-91c8-c7bda5abb97e None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:10.720147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2c7b6-7ccf-48ac-91c8-c7bda5abb97e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:10.720457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2c7b6-7ccf-48ac-91c8-c7bda5abb97e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:10.721799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01c5d4f-f51b-4bc7-87c8-02b34e0aede3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:10.721930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b2c7b6-7ccf-48ac-91c8-c7bda5abb97e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:10.728766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 362/714] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 10 14:01:10 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:10.729749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2c7b6-7ccf-48ac-91c8-c7bda5abb97e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:10.761085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4aae9-3d16-4c6d-a40e-ce1f1b7377a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:10.761712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4aae9-3d16-4c6d-a40e-ce1f1b7377a0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:10.761987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4aae9-3d16-4c6d-a40e-ce1f1b7377a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:10.762305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4aae9-3d16-4c6d-a40e-ce1f1b7377a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:10.763079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4aae9-3d16-4c6d-a40e-ce1f1b7377a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:10.763503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4aae9-3d16-4c6d-a40e-ce1f1b7377a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:10.765085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4aae9-3d16-4c6d-a40e-ce1f1b7377a0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:10.780732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 353/715] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:10.786707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:10.788059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4aae9-3d16-4c6d-a40e-ce1f1b7377a0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:10.849653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b2c7b6-7ccf-48ac-91c8-c7bda5abb97e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:10.889716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37381fb0-0d2c-40a9-bfda-0809756ab04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:10.890344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37381fb0-0d2c-40a9-bfda-0809756ab04e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:10.890617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37381fb0-0d2c-40a9-bfda-0809756ab04e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:10.891034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37381fb0-0d2c-40a9-bfda-0809756ab04e None admin] PATH_INFO: `/v3/services/network` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:10.891649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37381fb0-0d2c-40a9-bfda-0809756ab04e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:10.893045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37381fb0-0d2c-40a9-bfda-0809756ab04e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:10.904643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4aae9-3d16-4c6d-a40e-ce1f1b7377a0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-37381fb0-0d2c-40a9-bfda-0809756ab04e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:10.905215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:10.907246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:10.908792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 363/716] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:10 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:10.919088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37381fb0-0d2c-40a9-bfda-0809756ab04e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:10.976302 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a905533-7210-4910-8b4b-d27153eaa2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:10.976941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a905533-7210-4910-8b4b-d27153eaa2ee None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:10.977223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a905533-7210-4910-8b4b-d27153eaa2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:10.977502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a905533-7210-4910-8b4b-d27153eaa2ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:10.978270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a905533-7210-4910-8b4b-d27153eaa2ee None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=cb24a1b55ee7408ab24c68991aec54ec) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:10.978582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a905533-7210-4910-8b4b-d27153eaa2ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:10.980129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a905533-7210-4910-8b4b-d27153eaa2ee None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:10.989156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:10.997879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 354/717] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:10 2022] GET /identity/v3/role_assignments?user.id=cb24a1b55ee7408ab24c68991aec54ec&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:11.049024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37381fb0-0d2c-40a9-bfda-0809756ab04e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:11.060467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:11.102573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa26908-b3a4-4c5a-bc8f-09de09ced66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:11.102573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa26908-b3a4-4c5a-bc8f-09de09ced66e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:11.102573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa26908-b3a4-4c5a-bc8f-09de09ced66e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:11.102573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa26908-b3a4-4c5a-bc8f-09de09ced66e None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:11.103348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa26908-b3a4-4c5a-bc8f-09de09ced66e None admin] RBAC: Adding query filter params (name=network) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:11.103348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa26908-b3a4-4c5a-bc8f-09de09ced66e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:11.103348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa26908-b3a4-4c5a-bc8f-09de09ced66e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:11.108165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 364/718] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:11.116595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a905533-7210-4910-8b4b-d27153eaa2ee None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:11.262372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a905533-7210-4910-8b4b-d27153eaa2ee None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:11.311853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3ff508-cf83-4bc9-88ce-454e85a679a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:11.312468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff508-cf83-4bc9-88ce-454e85a679a4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:11.312745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff508-cf83-4bc9-88ce-454e85a679a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:11.313017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3ff508-cf83-4bc9-88ce-454e85a679a4 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:11.313656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ff508-cf83-4bc9-88ce-454e85a679a4 None admin] RBAC: Adding query filter params (type=network) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:11.313966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ff508-cf83-4bc9-88ce-454e85a679a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:11.315345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3ff508-cf83-4bc9-88ce-454e85a679a4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:11.321609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 355/719] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:11 2022] GET /identity/v3/services?type=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:11.756538 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e01e49-ffb1-4c5f-af29-3004640775b3 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:11.757473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e01e49-ffb1-4c5f-af29-3004640775b3 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:11.757962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e01e49-ffb1-4c5f-af29-3004640775b3 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:11.759526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 365/720] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:11.775340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42ee12-1f07-4c16-975e-1dacf2a705ba None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:11.776505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42ee12-1f07-4c16-975e-1dacf2a705ba None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:11.777024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42ee12-1f07-4c16-975e-1dacf2a705ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:11.802787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fe42ee12-1f07-4c16-975e-1dacf2a705ba None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:11.804336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fe42ee12-1f07-4c16-975e-1dacf2a705ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:11.887087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:11.901076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 356/721] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 10 14:01:12.191844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e01e49-ffb1-4c5f-af29-3004640775b3 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:12.332306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e01e49-ffb1-4c5f-af29-3004640775b3 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:12.374099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d655317-9dab-468a-b9e6-4fb513bd89a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:12.375128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d655317-9dab-468a-b9e6-4fb513bd89a2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:12.375637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d655317-9dab-468a-b9e6-4fb513bd89a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:12.376109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d655317-9dab-468a-b9e6-4fb513bd89a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:12.396420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d655317-9dab-468a-b9e6-4fb513bd89a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-7d655317-9dab-468a-b9e6-4fb513bd89a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:12.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:12.402561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:12.403402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:12.406036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 366/722] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:12.412519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42ee12-1f07-4c16-975e-1dacf2a705ba None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:12.572874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42ee12-1f07-4c16-975e-1dacf2a705ba None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:12.615820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07acfaa9-ab07-44c7-9676-91cbad6ebc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:12.616432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acfaa9-ab07-44c7-9676-91cbad6ebc2c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:12.616701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acfaa9-ab07-44c7-9676-91cbad6ebc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:12.616967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07acfaa9-ab07-44c7-9676-91cbad6ebc2c None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:12.617632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acfaa9-ab07-44c7-9676-91cbad6ebc2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:12.617925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acfaa9-ab07-44c7-9676-91cbad6ebc2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:12.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07acfaa9-ab07-44c7-9676-91cbad6ebc2c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:12.628289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 357/723] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:12.633660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d655317-9dab-468a-b9e6-4fb513bd89a2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:12.689807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafca1a4-30d2-41a2-b23c-cce358302f6f None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:12.690140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafca1a4-30d2-41a2-b23c-cce358302f6f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:12.690350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafca1a4-30d2-41a2-b23c-cce358302f6f None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:12.692123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:12.692403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 358/724] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:12.698662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e535e17-1ad5-424b-99ad-da5396b08534 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:12.699285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e535e17-1ad5-424b-99ad-da5396b08534 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:12.699831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e535e17-1ad5-424b-99ad-da5396b08534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:12.733188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4e535e17-1ad5-424b-99ad-da5396b08534 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:12.734719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4e535e17-1ad5-424b-99ad-da5396b08534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:12.767362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d655317-9dab-468a-b9e6-4fb513bd89a2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:12.804311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7176c3-85d2-4db6-bd03-aba29e90c4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:12.804938 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7176c3-85d2-4db6-bd03-aba29e90c4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:12.805212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7176c3-85d2-4db6-bd03-aba29e90c4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:12.805481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7176c3-85d2-4db6-bd03-aba29e90c4b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:12.815797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7176c3-85d2-4db6-bd03-aba29e90c4b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:12.817741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7176c3-85d2-4db6-bd03-aba29e90c4b8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:12.829610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-5f7176c3-85d2-4db6-bd03-aba29e90c4b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:12.833324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:12.834382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:12.835317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:12.835317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:12.835317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:12.839241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 367/725] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:12.840459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7176c3-85d2-4db6-bd03-aba29e90c4b8 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:12.870283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 359/726] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:12.926656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:12.943174 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7176c3-85d2-4db6-bd03-aba29e90c4b8 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:12.978593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c18f8-3a78-4679-bf29-fe87c2ceb410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:12.979195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c18f8-3a78-4679-bf29-fe87c2ceb410 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:12.979491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c18f8-3a78-4679-bf29-fe87c2ceb410 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:12.979764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2c18f8-3a78-4679-bf29-fe87c2ceb410 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:12.980397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c18f8-3a78-4679-bf29-fe87c2ceb410 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:12.980690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c18f8-3a78-4679-bf29-fe87c2ceb410 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:12.982057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2c18f8-3a78-4679-bf29-fe87c2ceb410 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:12.999160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 368/727] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:13.000575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e535e17-1ad5-424b-99ad-da5396b08534 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:13.119810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e535e17-1ad5-424b-99ad-da5396b08534 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:13.167524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01198a-b791-4c85-941b-89d706b3d869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:13.168491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01198a-b791-4c85-941b-89d706b3d869 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:13.169022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01198a-b791-4c85-941b-89d706b3d869 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:13.169461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01198a-b791-4c85-941b-89d706b3d869 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:13.182339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01198a-b791-4c85-941b-89d706b3d869 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:13.184152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01198a-b791-4c85-941b-89d706b3d869 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6f01198a-b791-4c85-941b-89d706b3d869 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:13.197060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:13.198009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:13.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:13.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:13.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:13.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:13.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:13.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:13.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:13.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:13.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:01:13.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:13.203667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 360/728] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:13.206980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c18f8-3a78-4679-bf29-fe87c2ceb410 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:13.309299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2c18f8-3a78-4679-bf29-fe87c2ceb410 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:13.321413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01198a-b791-4c85-941b-89d706b3d869 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:13.382226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eec113-0162-430f-9d60-cadafa27dee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:13.382752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eec113-0162-430f-9d60-cadafa27dee0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:13.383027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eec113-0162-430f-9d60-cadafa27dee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:13.383295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eec113-0162-430f-9d60-cadafa27dee0 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:13.384051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eec113-0162-430f-9d60-cadafa27dee0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:13.384352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eec113-0162-430f-9d60-cadafa27dee0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:13.385800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eec113-0162-430f-9d60-cadafa27dee0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:13.400099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:13.430905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 369/729] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 10 14:01:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:13.444070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eec113-0162-430f-9d60-cadafa27dee0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:13.449428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:13.516620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01198a-b791-4c85-941b-89d706b3d869 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:13.571585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3924cb-aba4-4d84-adcc-ec82f2ce66fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:13.572232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3924cb-aba4-4d84-adcc-ec82f2ce66fd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:13.572560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3924cb-aba4-4d84-adcc-ec82f2ce66fd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:13.572848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3924cb-aba4-4d84-adcc-ec82f2ce66fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:13.584648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3924cb-aba4-4d84-adcc-ec82f2ce66fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2e3924cb-aba4-4d84-adcc-ec82f2ce66fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:13.590289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:13.591214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:13.592073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:13.596544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 361/730] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:01:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:13.597959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86c370-9f2e-4501-a540-7ea3a54ad7cb None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:13.598241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86c370-9f2e-4501-a540-7ea3a54ad7cb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:13.598499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86c370-9f2e-4501-a540-7ea3a54ad7cb None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:13.600164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 362/731] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:13.600720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86c370-9f2e-4501-a540-7ea3a54ad7cb None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:13.603576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eec113-0162-430f-9d60-cadafa27dee0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:13.650229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f131d12-3ed4-4116-a9b9-756d2eb096bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:13.651407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f131d12-3ed4-4116-a9b9-756d2eb096bf None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:13.651933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f131d12-3ed4-4116-a9b9-756d2eb096bf None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:13.652743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f131d12-3ed4-4116-a9b9-756d2eb096bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:13.746127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86c370-9f2e-4501-a540-7ea3a54ad7cb None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:13.747338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f131d12-3ed4-4116-a9b9-756d2eb096bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:13.750260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f131d12-3ed4-4116-a9b9-756d2eb096bf None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:13.752695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:13.756322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:13.789562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115a3289-ebfa-4a21-bf65-1526df1ce64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:13.790136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3289-ebfa-4a21-bf65-1526df1ce64d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:13.790417 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3289-ebfa-4a21-bf65-1526df1ce64d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:13.790690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115a3289-ebfa-4a21-bf65-1526df1ce64d None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:13.791355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a3289-ebfa-4a21-bf65-1526df1ce64d None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:13.791684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a3289-ebfa-4a21-bf65-1526df1ce64d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:13.793057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115a3289-ebfa-4a21-bf65-1526df1ce64d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:13.806210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 363/732] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:01:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:13.806210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccdc25-a8a9-46e9-a8dd-c71d601037e2 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:13.806210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccdc25-a8a9-46e9-a8dd-c71d601037e2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:13.806210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccccdc25-a8a9-46e9-a8dd-c71d601037e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:13.821917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ccccdc25-a8a9-46e9-a8dd-c71d601037e2 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:13.823107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ccccdc25-a8a9-46e9-a8dd-c71d601037e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:13.878152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 370/733] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 10 14:01:13.878152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5477d5b-ac63-4b19-a5ee-9c1db7f64f07 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:13.878152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5477d5b-ac63-4b19-a5ee-9c1db7f64f07 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:13.878152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5477d5b-ac63-4b19-a5ee-9c1db7f64f07 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:13.878152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 371/734] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:13.878152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5477d5b-ac63-4b19-a5ee-9c1db7f64f07 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:13.911555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 364/735] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:13.911555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccdc25-a8a9-46e9-a8dd-c71d601037e2 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:14.006796 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5477d5b-ac63-4b19-a5ee-9c1db7f64f07 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:14.008676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:14.028876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:14.033356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccccdc25-a8a9-46e9-a8dd-c71d601037e2 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:14.048806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1a5133-dd7e-4013-82a4-c5f3116cdd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:14.049535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a5133-dd7e-4013-82a4-c5f3116cdd2e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.049869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a5133-dd7e-4013-82a4-c5f3116cdd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.050199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1a5133-dd7e-4013-82a4-c5f3116cdd2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.063306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a5133-dd7e-4013-82a4-c5f3116cdd2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:14.065338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1a5133-dd7e-4013-82a4-c5f3116cdd2e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:14.082128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e8c07-f9e9-42b4-a992-3d0d81b1d1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:14.083132 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8c07-f9e9-42b4-a992-3d0d81b1d1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.087741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8c07-f9e9-42b4-a992-3d0d81b1d1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.088399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e8c07-f9e9-42b4-a992-3d0d81b1d1d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-3e1a5133-dd7e-4013-82a4-c5f3116cdd2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:14.088598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:14.089581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:14.090493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:14.090493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:14.090493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:14.094689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 372/736] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:14.096287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655028c-a323-4ba1-981c-13012a6f9611 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.096567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655028c-a323-4ba1-981c-13012a6f9611 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.096836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1655028c-a323-4ba1-981c-13012a6f9611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.104677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e8c07-f9e9-42b4-a992-3d0d81b1d1d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:14.107226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e8c07-f9e9-42b4-a992-3d0d81b1d1d4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:14.118157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-1655028c-a323-4ba1-981c-13012a6f9611 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:14.119473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1655028c-a323-4ba1-981c-13012a6f9611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-f88e8c07-f9e9-42b4-a992-3d0d81b1d1d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:14.125424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:14.126344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:14.127211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:14.127211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:14.127211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:14.129278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 365/737] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:14.130685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e8c07-f9e9-42b4-a992-3d0d81b1d1d4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:14.197902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 373/738] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:14.198860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655028c-a323-4ba1-981c-13012a6f9611 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:14.245359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e8c07-f9e9-42b4-a992-3d0d81b1d1d4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:14.283940 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a98a99-2358-43cd-83f1-afe4aa7ccfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:14.284540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a98a99-2358-43cd-83f1-afe4aa7ccfd4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.285344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a98a99-2358-43cd-83f1-afe4aa7ccfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.285624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a98a99-2358-43cd-83f1-afe4aa7ccfd4 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.286261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a98a99-2358-43cd-83f1-afe4aa7ccfd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:14.286880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a98a99-2358-43cd-83f1-afe4aa7ccfd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:14.288371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a98a99-2358-43cd-83f1-afe4aa7ccfd4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:14.292807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:14.300229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 366/739] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:14.302758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77575-1c1e-4f43-8da3-62f53adb8b8c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.303227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77575-1c1e-4f43-8da3-62f53adb8b8c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.303712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a77575-1c1e-4f43-8da3-62f53adb8b8c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.305501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 367/740] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:14.306202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a77575-1c1e-4f43-8da3-62f53adb8b8c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:14.319798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:14.320960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1655028c-a323-4ba1-981c-13012a6f9611 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:14.358124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06fa68-f5b7-4cf3-b0c4-70167dc99738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:14.358730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06fa68-f5b7-4cf3-b0c4-70167dc99738 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.359008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06fa68-f5b7-4cf3-b0c4-70167dc99738 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.359276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b06fa68-f5b7-4cf3-b0c4-70167dc99738 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.359944 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06fa68-f5b7-4cf3-b0c4-70167dc99738 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:14.360246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06fa68-f5b7-4cf3-b0c4-70167dc99738 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:14.361605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b06fa68-f5b7-4cf3-b0c4-70167dc99738 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:14.373189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 374/741] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:14.373980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06fa68-f5b7-4cf3-b0c4-70167dc99738 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:14.421713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a77575-1c1e-4f43-8da3-62f53adb8b8c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:14.461233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec496ad-ea67-443e-9c2c-7a06fc579a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:14.462195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec496ad-ea67-443e-9c2c-7a06fc579a13 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.462630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec496ad-ea67-443e-9c2c-7a06fc579a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.462903 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec496ad-ea67-443e-9c2c-7a06fc579a13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.474727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec496ad-ea67-443e-9c2c-7a06fc579a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-cec496ad-ea67-443e-9c2c-7a06fc579a13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:14.479589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:14.480547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:14.481650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 368/742] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:14.482536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63795da1-1651-484d-b1d4-3e6970049aed None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.482536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63795da1-1651-484d-b1d4-3e6970049aed None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.482536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63795da1-1651-484d-b1d4-3e6970049aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.488863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b06fa68-f5b7-4cf3-b0c4-70167dc99738 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:14.499909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-63795da1-1651-484d-b1d4-3e6970049aed None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:14.501088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-63795da1-1651-484d-b1d4-3e6970049aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:14.523518 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8dff9-5309-4a49-b3a4-8a5f5796ab3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:14.524145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8dff9-5309-4a49-b3a4-8a5f5796ab3e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.524420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8dff9-5309-4a49-b3a4-8a5f5796ab3e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.524689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b8dff9-5309-4a49-b3a4-8a5f5796ab3e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.532052 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8dff9-5309-4a49-b3a4-8a5f5796ab3e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:14.533478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b8dff9-5309-4a49-b3a4-8a5f5796ab3e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-f8b8dff9-5309-4a49-b3a4-8a5f5796ab3e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:14.544307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:14.545273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 375/743] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 10 14:01:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:14.546210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8dff9-5309-4a49-b3a4-8a5f5796ab3e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:14.550369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:14.568559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:14.573217 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 369/744] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 10 14:01:14.574233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63795da1-1651-484d-b1d4-3e6970049aed None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:14.659094 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b8dff9-5309-4a49-b3a4-8a5f5796ab3e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:14.678042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63795da1-1651-484d-b1d4-3e6970049aed None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:14.694814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d5e1b-2f41-4bd8-93c1-b5de08b90a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:14.695463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d5e1b-2f41-4bd8-93c1-b5de08b90a47 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.695756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d5e1b-2f41-4bd8-93c1-b5de08b90a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.696029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d5e1b-2f41-4bd8-93c1-b5de08b90a47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.702698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d5e1b-2f41-4bd8-93c1-b5de08b90a47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:14.704188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d5e1b-2f41-4bd8-93c1-b5de08b90a47 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:14.714033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa1469-bbab-46f3-89b0-1ca0182b6608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:14.714575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa1469-bbab-46f3-89b0-1ca0182b6608 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.714851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa1469-bbab-46f3-89b0-1ca0182b6608 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.715122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa1469-bbab-46f3-89b0-1ca0182b6608 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.715801 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa1469-bbab-46f3-89b0-1ca0182b6608 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-337d5e1b-2f41-4bd8-93c1-b5de08b90a47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:14.716331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:14.717167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:14.718133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:14.718133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:14.718133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:14.718133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:14.718133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:14.718133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:14.718133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:14.718133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:14.718553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa1469-bbab-46f3-89b0-1ca0182b6608 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:14.723046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 376/745] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:14.724013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d5e1b-2f41-4bd8-93c1-b5de08b90a47 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:14.736187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 370/746] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:01:14 2022] POST /identity/v3/services => generated 251 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:14.737265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa1469-bbab-46f3-89b0-1ca0182b6608 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:14.810537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:14.828478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:14.869542 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa1469-bbab-46f3-89b0-1ca0182b6608 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:14.884293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d5e1b-2f41-4bd8-93c1-b5de08b90a47 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:14.906433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de1901-f026-4a76-9f37-0c1b9bfbc693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:14.907381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de1901-f026-4a76-9f37-0c1b9bfbc693 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.907381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de1901-f026-4a76-9f37-0c1b9bfbc693 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.907595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de1901-f026-4a76-9f37-0c1b9bfbc693 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.908332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de1901-f026-4a76-9f37-0c1b9bfbc693 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:14.908635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de1901-f026-4a76-9f37-0c1b9bfbc693 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:14.910217 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de1901-f026-4a76-9f37-0c1b9bfbc693 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:14.924043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46328c8-2514-4525-b5c9-b55049a67d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:14.924354 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46328c8-2514-4525-b5c9-b55049a67d5f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:14.924541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46328c8-2514-4525-b5c9-b55049a67d5f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:14.924812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46328c8-2514-4525-b5c9-b55049a67d5f None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:14.925478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46328c8-2514-4525-b5c9-b55049a67d5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:14.926308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46328c8-2514-4525-b5c9-b55049a67d5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:14.928337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46328c8-2514-4525-b5c9-b55049a67d5f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:14.936333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 371/747] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 10 14:01:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:14.936333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de1901-f026-4a76-9f37-0c1b9bfbc693 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:14.939966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 377/748] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:01:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:14.940086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46328c8-2514-4525-b5c9-b55049a67d5f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:15.042272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de1901-f026-4a76-9f37-0c1b9bfbc693 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:15.059652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:15.083945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44bec8-a767-4a1a-bad9-3093b204f7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:15.084556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44bec8-a767-4a1a-bad9-3093b204f7dd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:15.084925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44bec8-a767-4a1a-bad9-3093b204f7dd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:15.085199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44bec8-a767-4a1a-bad9-3093b204f7dd None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:15.085912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44bec8-a767-4a1a-bad9-3093b204f7dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:15.086215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44bec8-a767-4a1a-bad9-3093b204f7dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:15.087728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44bec8-a767-4a1a-bad9-3093b204f7dd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:15.106016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 372/749] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:15.106016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44bec8-a767-4a1a-bad9-3093b204f7dd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:15.163119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46328c8-2514-4525-b5c9-b55049a67d5f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:15.190191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:15.199755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454e41f-00fb-4090-9453-d575f72b338b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:15.200621 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454e41f-00fb-4090-9453-d575f72b338b None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:15.200953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454e41f-00fb-4090-9453-d575f72b338b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:15.201223 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d454e41f-00fb-4090-9453-d575f72b338b None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:15.201887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454e41f-00fb-4090-9453-d575f72b338b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:15.203256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d454e41f-00fb-4090-9453-d575f72b338b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:15.216779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44bec8-a767-4a1a-bad9-3093b204f7dd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:15.219754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 378/750] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:01:14 2022] POST /identity/v3/services => generated 240 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:15.220820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454e41f-00fb-4090-9453-d575f72b338b None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:15.279351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a988f59-d642-4605-bba5-ec87a175135f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:15.280460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a988f59-d642-4605-bba5-ec87a175135f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:15.280939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a988f59-d642-4605-bba5-ec87a175135f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:15.281440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a988f59-d642-4605-bba5-ec87a175135f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:15.345436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:15.350985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d454e41f-00fb-4090-9453-d575f72b338b None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:15.386907 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a988f59-d642-4605-bba5-ec87a175135f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:15.388445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a988f59-d642-4605-bba5-ec87a175135f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:15.390119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cc137-1743-4f65-acfd-eca56f2eee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:15.390979 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cc137-1743-4f65-acfd-eca56f2eee57 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:15.391497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cc137-1743-4f65-acfd-eca56f2eee57 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:15.391990 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cc137-1743-4f65-acfd-eca56f2eee57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:15.402849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cc137-1743-4f65-acfd-eca56f2eee57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:15.404751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cc137-1743-4f65-acfd-eca56f2eee57 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-135cc137-1743-4f65-acfd-eca56f2eee57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:15.420421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:15.421598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:15.422692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:15.422692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:15.422692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:15.427420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 379/751] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:15.427420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cc137-1743-4f65-acfd-eca56f2eee57 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:15.482246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:15.523523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 373/752] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:01:15.524760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a988f59-d642-4605-bba5-ec87a175135f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:15.559905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cc137-1743-4f65-acfd-eca56f2eee57 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:15.607728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a578b2a-f641-435a-961e-b70e3bfec3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:15.608462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a578b2a-f641-435a-961e-b70e3bfec3c8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:15.608590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a578b2a-f641-435a-961e-b70e3bfec3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:15.608778 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a578b2a-f641-435a-961e-b70e3bfec3c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:15.609638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a578b2a-f641-435a-961e-b70e3bfec3c8 None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=bc0a7e1b772b4703ac2e5ec33b5ecf6b) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:15.609868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a578b2a-f641-435a-961e-b70e3bfec3c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:15.611319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a578b2a-f641-435a-961e-b70e3bfec3c8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:15.618032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 380/753] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:15 2022] GET /identity/v3/role_assignments?user.id=bc0a7e1b772b4703ac2e5ec33b5ecf6b&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:15.618846 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a578b2a-f641-435a-961e-b70e3bfec3c8 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:15.623605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:15.671653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a988f59-d642-4605-bba5-ec87a175135f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:15.727781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef4514-48d5-44a1-88fe-29b2e1d21ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:15.729097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef4514-48d5-44a1-88fe-29b2e1d21ebc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:15.729355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef4514-48d5-44a1-88fe-29b2e1d21ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:15.729628 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef4514-48d5-44a1-88fe-29b2e1d21ebc None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:15.730304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef4514-48d5-44a1-88fe-29b2e1d21ebc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:15.733445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef4514-48d5-44a1-88fe-29b2e1d21ebc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:15.734733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef4514-48d5-44a1-88fe-29b2e1d21ebc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:15.751725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a578b2a-f641-435a-961e-b70e3bfec3c8 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:15.753607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:15.763299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 374/754] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:15.769240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef4514-48d5-44a1-88fe-29b2e1d21ebc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:15.825337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41d51c-0292-4af2-be00-807921005a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:15.826858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41d51c-0292-4af2-be00-807921005a68 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:15.827135 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41d51c-0292-4af2-be00-807921005a68 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:15.827403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab41d51c-0292-4af2-be00-807921005a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:15.851921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41d51c-0292-4af2-be00-807921005a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:15.853402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab41d51c-0292-4af2-be00-807921005a68 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-ab41d51c-0292-4af2-be00-807921005a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:15.873870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:15.874816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:15.876652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:15.876652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:15.876652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:15.876652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 381/755] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:15.882037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41d51c-0292-4af2-be00-807921005a68 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:15.933469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef4514-48d5-44a1-88fe-29b2e1d21ebc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:15.975827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:15.986312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d90506-271e-46ae-a91c-9f13619414f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:15.987044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d90506-271e-46ae-a91c-9f13619414f4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:15.987333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d90506-271e-46ae-a91c-9f13619414f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:15.987634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d90506-271e-46ae-a91c-9f13619414f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:15.992678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab41d51c-0292-4af2-be00-807921005a68 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:16.001727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d90506-271e-46ae-a91c-9f13619414f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:16.003413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d90506-271e-46ae-a91c-9f13619414f4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-86d90506-271e-46ae-a91c-9f13619414f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:16.030337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:16.031312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:16.032727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 375/756] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 10 14:01:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:16.035516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d017a88-16f5-46cc-a2e0-d5a66b4c8e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:16.038804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d017a88-16f5-46cc-a2e0-d5a66b4c8e9e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:16.038804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d017a88-16f5-46cc-a2e0-d5a66b4c8e9e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:16.038804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d017a88-16f5-46cc-a2e0-d5a66b4c8e9e None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:16.038804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d017a88-16f5-46cc-a2e0-d5a66b4c8e9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:16.038804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d017a88-16f5-46cc-a2e0-d5a66b4c8e9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:16.039180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d017a88-16f5-46cc-a2e0-d5a66b4c8e9e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:16.040637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d90506-271e-46ae-a91c-9f13619414f4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:16.072624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 382/757] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:16.074734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d017a88-16f5-46cc-a2e0-d5a66b4c8e9e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:16.098359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:16.185825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d017a88-16f5-46cc-a2e0-d5a66b4c8e9e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:16.221423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d945e-c901-473e-be68-bdf2dfee0b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:16.222176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d945e-c901-473e-be68-bdf2dfee0b7d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:16.223036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d945e-c901-473e-be68-bdf2dfee0b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:16.223383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d945e-c901-473e-be68-bdf2dfee0b7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:16.230341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d945e-c901-473e-be68-bdf2dfee0b7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:16.231911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d945e-c901-473e-be68-bdf2dfee0b7d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:16.232713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d90506-271e-46ae-a91c-9f13619414f4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-559d945e-c901-473e-be68-bdf2dfee0b7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:16.243821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:16.244711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:16.245916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:16.245916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:16.245916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:16.245916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:16.245916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:16.245916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:16.245916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:16.245916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:16.245916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 383/758] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:16.250845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d945e-c901-473e-be68-bdf2dfee0b7d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:16.292374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd23f7-363e-4182-ba22-d385517e55e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:16.293075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd23f7-363e-4182-ba22-d385517e55e9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:16.293352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd23f7-363e-4182-ba22-d385517e55e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:16.293620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd23f7-363e-4182-ba22-d385517e55e9 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:16.294346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd23f7-363e-4182-ba22-d385517e55e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:16.294641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd23f7-363e-4182-ba22-d385517e55e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:16.296249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd23f7-363e-4182-ba22-d385517e55e9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:16.317758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 376/759] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 10 14:01:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:16.325517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd23f7-363e-4182-ba22-d385517e55e9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:16.331149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:16.358537 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d945e-c901-473e-be68-bdf2dfee0b7d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:16.393708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816f202-510d-4dcd-8380-57fbdfa22ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:16.394826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f202-510d-4dcd-8380-57fbdfa22ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:16.395127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f202-510d-4dcd-8380-57fbdfa22ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:16.395466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816f202-510d-4dcd-8380-57fbdfa22ff3 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:16.396170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816f202-510d-4dcd-8380-57fbdfa22ff3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:16.396471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816f202-510d-4dcd-8380-57fbdfa22ff3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:16.397898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816f202-510d-4dcd-8380-57fbdfa22ff3 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:16.423964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 384/760] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:16.424547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:16.431162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816f202-510d-4dcd-8380-57fbdfa22ff3 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:16.491553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd23f7-363e-4182-ba22-d385517e55e9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:16.528044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618cf9-34db-49d0-8cd8-8aff3b4efa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:16.528795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618cf9-34db-49d0-8cd8-8aff3b4efa61 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:16.529057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618cf9-34db-49d0-8cd8-8aff3b4efa61 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:16.529327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b618cf9-34db-49d0-8cd8-8aff3b4efa61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:16.541712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618cf9-34db-49d0-8cd8-8aff3b4efa61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:16.543489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b618cf9-34db-49d0-8cd8-8aff3b4efa61 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2b618cf9-34db-49d0-8cd8-8aff3b4efa61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:16.559919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:16.560859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:16.561724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:16.561724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:16.561724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:16.563810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 377/761] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:16.570060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618cf9-34db-49d0-8cd8-8aff3b4efa61 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:16.623600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816f202-510d-4dcd-8380-57fbdfa22ff3 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:16.678308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f77d1a-f773-4a5d-aabb-aac183d3f936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:16.678918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f77d1a-f773-4a5d-aabb-aac183d3f936 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:16.680496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f77d1a-f773-4a5d-aabb-aac183d3f936 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:16.680766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f77d1a-f773-4a5d-aabb-aac183d3f936 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:16.681498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f77d1a-f773-4a5d-aabb-aac183d3f936 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=d75640e511e64661826c8b8d7aac2ce7) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:16.681800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f77d1a-f773-4a5d-aabb-aac183d3f936 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:16.683220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f77d1a-f773-4a5d-aabb-aac183d3f936 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:16.686513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:16.693740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 385/762] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:16 2022] GET /identity/v3/role_assignments?user.id=d75640e511e64661826c8b8d7aac2ce7&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 259 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:16.773544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:16.777224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b618cf9-34db-49d0-8cd8-8aff3b4efa61 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:16.833330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203fb02-61e0-4550-8c0f-55b55ea4e758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:16.833949 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203fb02-61e0-4550-8c0f-55b55ea4e758 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:16.834220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203fb02-61e0-4550-8c0f-55b55ea4e758 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:16.834484 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4203fb02-61e0-4550-8c0f-55b55ea4e758 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:16.835130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203fb02-61e0-4550-8c0f-55b55ea4e758 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:16.835433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203fb02-61e0-4550-8c0f-55b55ea4e758 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:16.836831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4203fb02-61e0-4550-8c0f-55b55ea4e758 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:16.856012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 378/763] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:16.861649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f77d1a-f773-4a5d-aabb-aac183d3f936 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:16.993284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f77d1a-f773-4a5d-aabb-aac183d3f936 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:17.029353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df9a61a-4d4c-424a-952c-dd967d35d999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:17.029908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9a61a-4d4c-424a-952c-dd967d35d999 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:17.030183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9a61a-4d4c-424a-952c-dd967d35d999 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:17.030452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df9a61a-4d4c-424a-952c-dd967d35d999 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:17.038001 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9a61a-4d4c-424a-952c-dd967d35d999 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:17.039606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df9a61a-4d4c-424a-952c-dd967d35d999 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-0df9a61a-4d4c-424a-952c-dd967d35d999 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:17.051714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:17.052732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:17.053591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:17.053591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:17.053591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:17.053591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:17.053591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:17.053591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:17.053591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:17.053591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:17.056025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 386/764] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:17.061467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203fb02-61e0-4550-8c0f-55b55ea4e758 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:17.074743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf2b4d-2b6d-4415-9838-42615a37cd8d None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:17.075045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf2b4d-2b6d-4415-9838-42615a37cd8d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:17.075286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf2b4d-2b6d-4415-9838-42615a37cd8d None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:17.076902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 387/765] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:01:17.082435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102302c9-b771-4223-a444-74d4f8c3f8fc None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:17.082715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102302c9-b771-4223-a444-74d4f8c3f8fc None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:17.082975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102302c9-b771-4223-a444-74d4f8c3f8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:17.115156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-102302c9-b771-4223-a444-74d4f8c3f8fc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:17.116384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-102302c9-b771-4223-a444-74d4f8c3f8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:17.215506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:17.244181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 388/766] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:17.247477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4203fb02-61e0-4550-8c0f-55b55ea4e758 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:17.283680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183b0c8-be6f-4e09-bf55-c1c2cfbae764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:17.284613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183b0c8-be6f-4e09-bf55-c1c2cfbae764 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:17.285334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183b0c8-be6f-4e09-bf55-c1c2cfbae764 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:17.286023 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c183b0c8-be6f-4e09-bf55-c1c2cfbae764 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:17.286930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183b0c8-be6f-4e09-bf55-c1c2cfbae764 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:17.287428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183b0c8-be6f-4e09-bf55-c1c2cfbae764 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:17.289174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c183b0c8-be6f-4e09-bf55-c1c2cfbae764 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:17.303060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 379/767] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:17.308630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102302c9-b771-4223-a444-74d4f8c3f8fc None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:17.446115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102302c9-b771-4223-a444-74d4f8c3f8fc None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:17.482608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affb834-70b9-429c-8b8e-0ee72c818fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:17.483703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affb834-70b9-429c-8b8e-0ee72c818fea None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:17.484184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affb834-70b9-429c-8b8e-0ee72c818fea None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:17.484766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affb834-70b9-429c-8b8e-0ee72c818fea None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/cb24a1b55ee7408ab24c68991aec54ec/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:17.486893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affb834-70b9-429c-8b8e-0ee72c818fea None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=cb24a1b55ee7408ab24c68991aec54ec, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:17.488803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affb834-70b9-429c-8b8e-0ee72c818fea None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:17.511058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 389/768] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:01:17 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/cb24a1b55ee7408ab24c68991aec54ec/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:01:17.593153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183b0c8-be6f-4e09-bf55-c1c2cfbae764 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:17.622272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:17.726596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c183b0c8-be6f-4e09-bf55-c1c2cfbae764 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:17.773335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72747755-425d-4f7e-818b-faa5796e4513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:17.774346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72747755-425d-4f7e-818b-faa5796e4513 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:17.774821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72747755-425d-4f7e-818b-faa5796e4513 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:17.775380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72747755-425d-4f7e-818b-faa5796e4513 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:17.776179 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72747755-425d-4f7e-818b-faa5796e4513 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:17.777799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72747755-425d-4f7e-818b-faa5796e4513 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-72747755-425d-4f7e-818b-faa5796e4513 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:17.794377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:17.795437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 14:01:17.796340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:17.799365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 380/769] 10.222.0.26 () {56 vars in 1129 bytes} [Sat Sep 10 14:01:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:17.805478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affb834-70b9-429c-8b8e-0ee72c818fea None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:17.894843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:17.921695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dba4a-4cf1-4ee4-a99d-da0b96bdd6c3 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:17.922038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dba4a-4cf1-4ee4-a99d-da0b96bdd6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:17.922245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dba4a-4cf1-4ee4-a99d-da0b96bdd6c3 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:17.924558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 381/770] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:17.930656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b1b56-f1e7-4e67-b874-d2d01cf34661 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:17.931226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b1b56-f1e7-4e67-b874-d2d01cf34661 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:17.931750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b1b56-f1e7-4e67-b874-d2d01cf34661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:17.959057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-166b1b56-f1e7-4e67-b874-d2d01cf34661 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:17.960646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-166b1b56-f1e7-4e67-b874-d2d01cf34661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:17.982702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affb834-70b9-429c-8b8e-0ee72c818fea None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:18.026260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53efaa-d637-493b-9553-eaa9adb09619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:18.027969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53efaa-d637-493b-9553-eaa9adb09619 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:18.028448 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53efaa-d637-493b-9553-eaa9adb09619 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:18.029320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f53efaa-d637-493b-9553-eaa9adb09619 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:18.030428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53efaa-d637-493b-9553-eaa9adb09619 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:18.031026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53efaa-d637-493b-9553-eaa9adb09619 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:18.032731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f53efaa-d637-493b-9553-eaa9adb09619 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:18.038512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 390/772] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 10 14:01:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:18.039245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 382/772] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:18.043623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53efaa-d637-493b-9553-eaa9adb09619 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:18.160834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f53efaa-d637-493b-9553-eaa9adb09619 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:18.174978 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:18.200321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a62a5-92a5-4e32-a375-6df4840633ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:18.201347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a62a5-92a5-4e32-a375-6df4840633ad None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:18.201825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a62a5-92a5-4e32-a375-6df4840633ad None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:18.202305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a62a5-92a5-4e32-a375-6df4840633ad None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:18.203131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a62a5-92a5-4e32-a375-6df4840633ad None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:18.203675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a62a5-92a5-4e32-a375-6df4840633ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:18.205229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a62a5-92a5-4e32-a375-6df4840633ad None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:18.211090 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 391/773] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 10 14:01:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:18.216577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b1b56-f1e7-4e67-b874-d2d01cf34661 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:18.244901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:18.363255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b1b56-f1e7-4e67-b874-d2d01cf34661 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:18.407296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826435a-21e1-4e3f-b8ac-c0e60c7c80f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:18.407957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826435a-21e1-4e3f-b8ac-c0e60c7c80f3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:18.408241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826435a-21e1-4e3f-b8ac-c0e60c7c80f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:18.408515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826435a-21e1-4e3f-b8ac-c0e60c7c80f3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:18.409225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826435a-21e1-4e3f-b8ac-c0e60c7c80f3 None admin] RBAC: Adding query filter params (interface=public, service_id=dbbcea06598543c786f41067c623053c, region_id=RegionOne) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:18.409539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826435a-21e1-4e3f-b8ac-c0e60c7c80f3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:18.411523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826435a-21e1-4e3f-b8ac-c0e60c7c80f3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:18.413577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a62a5-92a5-4e32-a375-6df4840633ad None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:18.422308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 383/774] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 10 14:01:18 2022] GET /identity/v3/endpoints?service_id=dbbcea06598543c786f41067c623053c&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:18.428084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826435a-21e1-4e3f-b8ac-c0e60c7c80f3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:18.584037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:18.606458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a62a5-92a5-4e32-a375-6df4840633ad None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:18.618317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826435a-21e1-4e3f-b8ac-c0e60c7c80f3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:18.656577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a59766-704f-4cd0-aabf-f06a7fb292fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:18.657613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a59766-704f-4cd0-aabf-f06a7fb292fe None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:18.658098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a59766-704f-4cd0-aabf-f06a7fb292fe None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:18.658661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a59766-704f-4cd0-aabf-f06a7fb292fe None admin] PATH_INFO: `/v3/services/network` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:18.659456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a59766-704f-4cd0-aabf-f06a7fb292fe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:18.661261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a59766-704f-4cd0-aabf-f06a7fb292fe None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-57a59766-704f-4cd0-aabf-f06a7fb292fe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:18.677640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:18.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 14:01:18.679472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:18.684323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 392/775] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:18 2022] GET /identity/v3/services/network => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:18.688088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6a59b-8f6c-4cdc-b935-0db9e179f611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:18.689241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6a59b-8f6c-4cdc-b935-0db9e179f611 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:18.689748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6a59b-8f6c-4cdc-b935-0db9e179f611 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:18.690030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3956a9f-9288-4b97-9328-5de4da8d72b4 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:18.690312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3956a9f-9288-4b97-9328-5de4da8d72b4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:18.690583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3956a9f-9288-4b97-9328-5de4da8d72b4 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:18.690835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6a59b-8f6c-4cdc-b935-0db9e179f611 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:18.692220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6a59b-8f6c-4cdc-b935-0db9e179f611 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:18.694078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6a59b-8f6c-4cdc-b935-0db9e179f611 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:18.695526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 393/776] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:01:18.697200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3956a9f-9288-4b97-9328-5de4da8d72b4 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:18.702824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 384/777] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 10 14:01:18 2022] GET /identity/v3/services => generated 772 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:18.708309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532e05-430b-4878-9831-7607a7de5025 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:18.710707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532e05-430b-4878-9831-7607a7de5025 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:18.711216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532e05-430b-4878-9831-7607a7de5025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:18.734735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-54532e05-430b-4878-9831-7607a7de5025 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:18.735991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-54532e05-430b-4878-9831-7607a7de5025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:18.788843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:18.853904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 385/778] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:18.876522 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3956a9f-9288-4b97-9328-5de4da8d72b4 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:18.913570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb718b0c-69c5-498f-ae66-74e42a6eb216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:18.914178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb718b0c-69c5-498f-ae66-74e42a6eb216 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:18.914455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb718b0c-69c5-498f-ae66-74e42a6eb216 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:18.914728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb718b0c-69c5-498f-ae66-74e42a6eb216 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:18.915362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb718b0c-69c5-498f-ae66-74e42a6eb216 None admin] RBAC: Adding query filter params (name=network) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:18.915700 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb718b0c-69c5-498f-ae66-74e42a6eb216 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:18.917096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb718b0c-69c5-498f-ae66-74e42a6eb216 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:18.922641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 394/779] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:18 2022] GET /identity/v3/services?name=network => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:18.928972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54532e05-430b-4878-9831-7607a7de5025 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:19.015706 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:19.046856 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54532e05-430b-4878-9831-7607a7de5025 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:19.083006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329db205-044f-4944-b980-36c91aa08c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:19.083691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329db205-044f-4944-b980-36c91aa08c4a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:19.084097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329db205-044f-4944-b980-36c91aa08c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:19.084371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329db205-044f-4944-b980-36c91aa08c4a None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:19.085113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329db205-044f-4944-b980-36c91aa08c4a None admin] RBAC: Adding query filter params (type=network) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:19.085415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329db205-044f-4944-b980-36c91aa08c4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:19.086782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329db205-044f-4944-b980-36c91aa08c4a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:19.092186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 386/780] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:18 2022] GET /identity/v3/services?type=network => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:19.098551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb718b0c-69c5-498f-ae66-74e42a6eb216 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:19.164914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329db205-044f-4944-b980-36c91aa08c4a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:19.223573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb718b0c-69c5-498f-ae66-74e42a6eb216 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:19.226651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:19.281230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada68b3-dce3-4d17-b90d-e5fe52bb9814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:19.281848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada68b3-dce3-4d17-b90d-e5fe52bb9814 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:19.282125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada68b3-dce3-4d17-b90d-e5fe52bb9814 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:19.282394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada68b3-dce3-4d17-b90d-e5fe52bb9814 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:19.283098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada68b3-dce3-4d17-b90d-e5fe52bb9814 None admin] RBAC: Adding query filter params (interface=public, service_id=d587698bf099479688030eb499978fc2, region_id=RegionOne) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:19.283409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada68b3-dce3-4d17-b90d-e5fe52bb9814 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:19.290016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada68b3-dce3-4d17-b90d-e5fe52bb9814 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:19.306424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 395/781] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 10 14:01:19 2022] GET /identity/v3/endpoints?service_id=d587698bf099479688030eb499978fc2&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:19.311529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada68b3-dce3-4d17-b90d-e5fe52bb9814 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:19.356597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329db205-044f-4944-b980-36c91aa08c4a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:19.404547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7357a446-d1c6-4c03-ad08-90cfdc77e69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:19.405145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357a446-d1c6-4c03-ad08-90cfdc77e69b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:19.405686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357a446-d1c6-4c03-ad08-90cfdc77e69b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:19.405965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7357a446-d1c6-4c03-ad08-90cfdc77e69b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:19.410068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:19.422870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7357a446-d1c6-4c03-ad08-90cfdc77e69b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-7357a446-d1c6-4c03-ad08-90cfdc77e69b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:19.428587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:19.429754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:19.430799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:19.432897 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 387/782] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:19.438740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5eae60-8d47-4950-8db7-40a445fe5262 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:19.439240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5eae60-8d47-4950-8db7-40a445fe5262 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:19.439419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada68b3-dce3-4d17-b90d-e5fe52bb9814 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:19.439930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5eae60-8d47-4950-8db7-40a445fe5262 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:19.444085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 388/783] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:19 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:19.445122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5eae60-8d47-4950-8db7-40a445fe5262 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:19.478202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea876d5f-8560-4bc0-a588-eaf6d9cc7632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:19.478723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876d5f-8560-4bc0-a588-eaf6d9cc7632 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:19.478997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876d5f-8560-4bc0-a588-eaf6d9cc7632 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:19.479263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea876d5f-8560-4bc0-a588-eaf6d9cc7632 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:19.479891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea876d5f-8560-4bc0-a588-eaf6d9cc7632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:19.481260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea876d5f-8560-4bc0-a588-eaf6d9cc7632 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:19.491623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 396/784] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 10 14:01:19 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:19.493172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d659d60-eb5f-46c3-b3a1-ac1c9b797496 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:19.493382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d659d60-eb5f-46c3-b3a1-ac1c9b797496 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:19.493658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d659d60-eb5f-46c3-b3a1-ac1c9b797496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:19.517375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-1d659d60-eb5f-46c3-b3a1-ac1c9b797496 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:19.518588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1d659d60-eb5f-46c3-b3a1-ac1c9b797496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:19.549838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:19.603797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 397/785] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:19.649282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5eae60-8d47-4950-8db7-40a445fe5262 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:19.693425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04235c5a-ac5d-4434-b7a4-6229a7fd8bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:19.694071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04235c5a-ac5d-4434-b7a4-6229a7fd8bec None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:19.694357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04235c5a-ac5d-4434-b7a4-6229a7fd8bec None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:19.694624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04235c5a-ac5d-4434-b7a4-6229a7fd8bec None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:19.695293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04235c5a-ac5d-4434-b7a4-6229a7fd8bec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:19.695615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04235c5a-ac5d-4434-b7a4-6229a7fd8bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:19.697016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04235c5a-ac5d-4434-b7a4-6229a7fd8bec None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:19.706630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 389/786] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:01:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:19.713242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d659d60-eb5f-46c3-b3a1-ac1c9b797496 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:19.826481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d659d60-eb5f-46c3-b3a1-ac1c9b797496 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:19.863785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0916668-f673-458f-964f-c7a45f83f7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:19.864863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0916668-f673-458f-964f-c7a45f83f7cf None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:19.865369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0916668-f673-458f-964f-c7a45f83f7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:19.865843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0916668-f673-458f-964f-c7a45f83f7cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:19.880094 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0916668-f673-458f-964f-c7a45f83f7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:19.881909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0916668-f673-458f-964f-c7a45f83f7cf None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-e0916668-f673-458f-964f-c7a45f83f7cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:19.901920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:19.902911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:19.903810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:19.903810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:19.903810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:19.905958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 398/787] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:19.911988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04235c5a-ac5d-4434-b7a4-6229a7fd8bec None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:19.914750 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0916668-f673-458f-964f-c7a45f83f7cf None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:19.939479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:20.017721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:20.061174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04235c5a-ac5d-4434-b7a4-6229a7fd8bec None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:20.099409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4428737-8768-4f2f-afc4-250a1291cfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:20.100397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4428737-8768-4f2f-afc4-250a1291cfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:20.101003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4428737-8768-4f2f-afc4-250a1291cfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:20.101506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4428737-8768-4f2f-afc4-250a1291cfa4 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:20.102395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4428737-8768-4f2f-afc4-250a1291cfa4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:20.102922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4428737-8768-4f2f-afc4-250a1291cfa4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:20.104546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4428737-8768-4f2f-afc4-250a1291cfa4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:20.118502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 390/788] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:20.124471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4428737-8768-4f2f-afc4-250a1291cfa4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:20.160758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0916668-f673-458f-964f-c7a45f83f7cf None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:20.232082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bc213-321d-4751-832c-51cfaf2557af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:20.232712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bc213-321d-4751-832c-51cfaf2557af None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:20.232986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bc213-321d-4751-832c-51cfaf2557af None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:20.233251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158bc213-321d-4751-832c-51cfaf2557af None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:20.255857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158bc213-321d-4751-832c-51cfaf2557af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-158bc213-321d-4751-832c-51cfaf2557af None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:20.261395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:20.262390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:20.263247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 399/789] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:01:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:20.272161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bc213-321d-4751-832c-51cfaf2557af None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:20.280717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4428737-8768-4f2f-afc4-250a1291cfa4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:20.317464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:20.353654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b3ec43-b989-4df6-a83a-122524f976de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:20.354212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3ec43-b989-4df6-a83a-122524f976de None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:20.354486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3ec43-b989-4df6-a83a-122524f976de None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:20.354754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b3ec43-b989-4df6-a83a-122524f976de None admin] PATH_INFO: `/v3/users/swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:20.364708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3ec43-b989-4df6-a83a-122524f976de None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:20.366183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b3ec43-b989-4df6-a83a-122524f976de None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-31b3ec43-b989-4df6-a83a-122524f976de None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:20.385319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:20.386578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:20.387565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 391/790] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:20 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:20.388162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e76f4-4b0a-4ead-8eeb-61060c50f56e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:20.388162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e76f4-4b0a-4ead-8eeb-61060c50f56e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:20.388661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e76f4-4b0a-4ead-8eeb-61060c50f56e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:20.390747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 392/791] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:20.392035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e76f4-4b0a-4ead-8eeb-61060c50f56e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:20.429122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:20.461508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158bc213-321d-4751-832c-51cfaf2557af None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:20.499724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c943b5-2888-4552-bdcf-997a0046d0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:20.500842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c943b5-2888-4552-bdcf-997a0046d0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:20.501323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c943b5-2888-4552-bdcf-997a0046d0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:20.501823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c943b5-2888-4552-bdcf-997a0046d0b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:20.502793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c943b5-2888-4552-bdcf-997a0046d0b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:20.503295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c943b5-2888-4552-bdcf-997a0046d0b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:20.505537 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c943b5-2888-4552-bdcf-997a0046d0b3 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:20.515011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 400/792] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:01:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:20.516644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ff6c0-5947-4dcf-8e50-8b3cd09b4be5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:20.517134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ff6c0-5947-4dcf-8e50-8b3cd09b4be5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:20.517616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ff6c0-5947-4dcf-8e50-8b3cd09b4be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:20.530065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e76f4-4b0a-4ead-8eeb-61060c50f56e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:20.537503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a27ff6c0-5947-4dcf-8e50-8b3cd09b4be5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:20.539086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a27ff6c0-5947-4dcf-8e50-8b3cd09b4be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:20.570623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a3f2a-60e7-46b3-9e21-cf2bf36a765a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:20.571028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a3f2a-60e7-46b3-9e21-cf2bf36a765a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:20.571311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a3f2a-60e7-46b3-9e21-cf2bf36a765a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:20.571718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6a3f2a-60e7-46b3-9e21-cf2bf36a765a None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:20.572448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a3f2a-60e7-46b3-9e21-cf2bf36a765a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:20.572805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a3f2a-60e7-46b3-9e21-cf2bf36a765a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:20.574325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6a3f2a-60e7-46b3-9e21-cf2bf36a765a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:20.594658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 393/793] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 10 14:01:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:20.595934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a3f2a-60e7-46b3-9e21-cf2bf36a765a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:20.623651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 401/794] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:20.624921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ff6c0-5947-4dcf-8e50-8b3cd09b4be5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:20.633863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:20.709027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:20.722068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6a3f2a-60e7-46b3-9e21-cf2bf36a765a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:20.750580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27ff6c0-5947-4dcf-8e50-8b3cd09b4be5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:20.759763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b8748-0a1e-44c1-9ef5-f76b6a309e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:20.760191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b8748-0a1e-44c1-9ef5-f76b6a309e4f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:20.760507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b8748-0a1e-44c1-9ef5-f76b6a309e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:20.760774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b8748-0a1e-44c1-9ef5-f76b6a309e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:20.772060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b8748-0a1e-44c1-9ef5-f76b6a309e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:20.773600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b8748-0a1e-44c1-9ef5-f76b6a309e4f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:20.790541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d39f4d-17ac-466f-a1d8-58c0e7747427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6b9b8748-0a1e-44c1-9ef5-f76b6a309e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:20.790832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:20.791800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:20.792668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:20.792668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:20.792668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:20.792668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 394/795] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:20.793985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d39f4d-17ac-466f-a1d8-58c0e7747427 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:20.794332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d39f4d-17ac-466f-a1d8-58c0e7747427 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:20.794671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d39f4d-17ac-466f-a1d8-58c0e7747427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:20.797486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b8748-0a1e-44c1-9ef5-f76b6a309e4f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:20.805976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d39f4d-17ac-466f-a1d8-58c0e7747427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:20.807809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d39f4d-17ac-466f-a1d8-58c0e7747427 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-37d39f4d-17ac-466f-a1d8-58c0e7747427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:20.822435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:20.823377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:20.824989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:20.824989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:20.824989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:20.824989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 402/796] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:20.836366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d39f4d-17ac-466f-a1d8-58c0e7747427 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:20.899068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:20.913041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b8748-0a1e-44c1-9ef5-f76b6a309e4f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:20.950221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c33ee-88c3-466f-a68c-335b2dd8c38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:20.951295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c33ee-88c3-466f-a68c-335b2dd8c38c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:20.951776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c33ee-88c3-466f-a68c-335b2dd8c38c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:20.952244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c33ee-88c3-466f-a68c-335b2dd8c38c None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:20.953065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c33ee-88c3-466f-a68c-335b2dd8c38c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:20.953638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c33ee-88c3-466f-a68c-335b2dd8c38c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:20.955305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9c33ee-88c3-466f-a68c-335b2dd8c38c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:20.969736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 395/797] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:20.975488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d39f4d-17ac-466f-a1d8-58c0e7747427 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:20.976705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c33ee-88c3-466f-a68c-335b2dd8c38c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:20.992831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:21.010115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4782d40-e535-4087-9462-a8a0687e7c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:21.010726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4782d40-e535-4087-9462-a8a0687e7c45 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.011007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4782d40-e535-4087-9462-a8a0687e7c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.011337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4782d40-e535-4087-9462-a8a0687e7c45 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.012137 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4782d40-e535-4087-9462-a8a0687e7c45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:21.012438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4782d40-e535-4087-9462-a8a0687e7c45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:21.013827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4782d40-e535-4087-9462-a8a0687e7c45 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:21.034557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 403/798] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:21.035902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4782d40-e535-4087-9462-a8a0687e7c45 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:21.123964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c33ee-88c3-466f-a68c-335b2dd8c38c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:21.160104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4782d40-e535-4087-9462-a8a0687e7c45 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:21.166278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93caf5-2338-4d8f-86e0-6f43703e7fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:21.166925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93caf5-2338-4d8f-86e0-6f43703e7fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.167198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93caf5-2338-4d8f-86e0-6f43703e7fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.167495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93caf5-2338-4d8f-86e0-6f43703e7fb4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.175310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93caf5-2338-4d8f-86e0-6f43703e7fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:21.177484 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93caf5-2338-4d8f-86e0-6f43703e7fb4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-0b93caf5-2338-4d8f-86e0-6f43703e7fb4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:21.192890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:21.194096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:21.195120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:21.195120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:21.195120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:21.195120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:21.195120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:21.195120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:21.195120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:21.195120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:21.195120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:21.195120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:21.198727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 396/799] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 10 14:01:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:21.199808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93caf5-2338-4d8f-86e0-6f43703e7fb4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:21.202958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2e5ec-cbbc-4e27-aa40-94aed82956d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:21.204486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2e5ec-cbbc-4e27-aa40-94aed82956d7 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.205849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2e5ec-cbbc-4e27-aa40-94aed82956d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.206317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c2e5ec-cbbc-4e27-aa40-94aed82956d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.214056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:21.223699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c2e5ec-cbbc-4e27-aa40-94aed82956d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-a8c2e5ec-cbbc-4e27-aa40-94aed82956d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:21.228609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:21.229526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:21.230352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:21.234659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 404/800] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:21.235538 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2e5ec-cbbc-4e27-aa40-94aed82956d7 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:21.281893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:21.331550 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93caf5-2338-4d8f-86e0-6f43703e7fb4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:21.364910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c2e5ec-cbbc-4e27-aa40-94aed82956d7 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:21.368721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734dc87-fb44-407d-b19b-fc75507c52f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:21.369401 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734dc87-fb44-407d-b19b-fc75507c52f2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.369681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734dc87-fb44-407d-b19b-fc75507c52f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.369950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1734dc87-fb44-407d-b19b-fc75507c52f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.377193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734dc87-fb44-407d-b19b-fc75507c52f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:21.380311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1734dc87-fb44-407d-b19b-fc75507c52f2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-1734dc87-fb44-407d-b19b-fc75507c52f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:21.394658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:21.395583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:21.396464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:21.396464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:21.396464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:21.396464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:21.396464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:21.396464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:21.396464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:21.396464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:21.400738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 397/801] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:21.401580 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734dc87-fb44-407d-b19b-fc75507c52f2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:21.408159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2685d4-88e3-4221-94b9-d63aeceff221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:21.410698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2685d4-88e3-4221-94b9-d63aeceff221 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.410924 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2685d4-88e3-4221-94b9-d63aeceff221 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.411199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2685d4-88e3-4221-94b9-d63aeceff221 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.411959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2685d4-88e3-4221-94b9-d63aeceff221 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:21.412314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2685d4-88e3-4221-94b9-d63aeceff221 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:21.413743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2685d4-88e3-4221-94b9-d63aeceff221 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:21.447771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 405/802] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 10 14:01:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:21.448750 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2685d4-88e3-4221-94b9-d63aeceff221 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:21.492991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:21.538092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1734dc87-fb44-407d-b19b-fc75507c52f2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:21.567392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:21.573943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aed9cb-99de-4362-a318-1b6d081bcc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:21.574447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2685d4-88e3-4221-94b9-d63aeceff221 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:21.574569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed9cb-99de-4362-a318-1b6d081bcc99 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.574818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed9cb-99de-4362-a318-1b6d081bcc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.575091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aed9cb-99de-4362-a318-1b6d081bcc99 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.575886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aed9cb-99de-4362-a318-1b6d081bcc99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:21.576090 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aed9cb-99de-4362-a318-1b6d081bcc99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:21.577474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aed9cb-99de-4362-a318-1b6d081bcc99 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:21.586469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 398/803] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:01:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:21.587298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aed9cb-99de-4362-a318-1b6d081bcc99 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:21.620931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012565e-d057-4639-8b4b-ceeacb845928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:21.621540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012565e-d057-4639-8b4b-ceeacb845928 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.621909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012565e-d057-4639-8b4b-ceeacb845928 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.622083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012565e-d057-4639-8b4b-ceeacb845928 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.622770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012565e-d057-4639-8b4b-ceeacb845928 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:21.623186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012565e-d057-4639-8b4b-ceeacb845928 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:21.624520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012565e-d057-4639-8b4b-ceeacb845928 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:21.642520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 406/804] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:21.642520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973b9a2-d761-46b3-87a8-23903c45341a None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.642520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973b9a2-d761-46b3-87a8-23903c45341a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.642520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2973b9a2-d761-46b3-87a8-23903c45341a None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.642520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 407/805] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:21.642520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973b9a2-d761-46b3-87a8-23903c45341a None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:21.702649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aed9cb-99de-4362-a318-1b6d081bcc99 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:21.732180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:21.741930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cb39ee-c069-44fb-9eed-4086815c4bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:21.742558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb39ee-c069-44fb-9eed-4086815c4bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.742829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb39ee-c069-44fb-9eed-4086815c4bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.743093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cb39ee-c069-44fb-9eed-4086815c4bd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.753846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb39ee-c069-44fb-9eed-4086815c4bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:21.755750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cb39ee-c069-44fb-9eed-4086815c4bd7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:21.758612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2973b9a2-d761-46b3-87a8-23903c45341a None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-25cb39ee-c069-44fb-9eed-4086815c4bd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:21.770724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:21.771919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:21.772797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:21.772797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:21.772797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:21.772797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 399/806] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:21.773017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec6d75-fce2-4a95-ac5c-51e4e9e375db None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.773840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec6d75-fce2-4a95-ac5c-51e4e9e375db None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.773840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec6d75-fce2-4a95-ac5c-51e4e9e375db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.792276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-27ec6d75-fce2-4a95-ac5c-51e4e9e375db None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:21.793465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-27ec6d75-fce2-4a95-ac5c-51e4e9e375db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:21.802151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970f871-97e8-4c1e-8bdf-a3092f973ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:21.802778 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970f871-97e8-4c1e-8bdf-a3092f973ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.803049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970f871-97e8-4c1e-8bdf-a3092f973ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.803361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6970f871-97e8-4c1e-8bdf-a3092f973ea0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.816837 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970f871-97e8-4c1e-8bdf-a3092f973ea0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:21.818251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6970f871-97e8-4c1e-8bdf-a3092f973ea0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-6970f871-97e8-4c1e-8bdf-a3092f973ea0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:21.834323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:21.835263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:21.840939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:21.840939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:21.840939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:21.840939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 408/807] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:21.840939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970f871-97e8-4c1e-8bdf-a3092f973ea0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:21.850410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:21.881070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 400/808] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:21.881070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec6d75-fce2-4a95-ac5c-51e4e9e375db None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:21.953596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6970f871-97e8-4c1e-8bdf-a3092f973ea0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:21.987951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec6d75-fce2-4a95-ac5c-51e4e9e375db None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:21.994121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5ccde-d705-4bab-8584-89ae76322202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:21.994801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ccde-d705-4bab-8584-89ae76322202 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:21.995076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ccde-d705-4bab-8584-89ae76322202 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:21.995345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b5ccde-d705-4bab-8584-89ae76322202 None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/bc0a7e1b772b4703ac2e5ec33b5ecf6b/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:21.997498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5ccde-d705-4bab-8584-89ae76322202 None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=bc0a7e1b772b4703ac2e5ec33b5ecf6b, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:21.999102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b5ccde-d705-4bab-8584-89ae76322202 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:22.005736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:22.023824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 409/809] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:01:21 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/bc0a7e1b772b4703ac2e5ec33b5ecf6b/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:01:22.024967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5ccde-d705-4bab-8584-89ae76322202 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:22.035681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292a7fd-04a4-44b7-8245-e8a70b737b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:22.036447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292a7fd-04a4-44b7-8245-e8a70b737b3d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.036731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292a7fd-04a4-44b7-8245-e8a70b737b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.037099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e292a7fd-04a4-44b7-8245-e8a70b737b3d None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.037805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292a7fd-04a4-44b7-8245-e8a70b737b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:22.038102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292a7fd-04a4-44b7-8245-e8a70b737b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:22.039781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e292a7fd-04a4-44b7-8245-e8a70b737b3d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:22.055547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 401/810] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:22.064823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292a7fd-04a4-44b7-8245-e8a70b737b3d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:22.146931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:22.187332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b5ccde-d705-4bab-8584-89ae76322202 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:22.203920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e292a7fd-04a4-44b7-8245-e8a70b737b3d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:22.224655 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb10dca7-e0da-4738-90f6-55d3ac1476e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:22.225421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10dca7-e0da-4738-90f6-55d3ac1476e6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.225606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10dca7-e0da-4738-90f6-55d3ac1476e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.225835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb10dca7-e0da-4738-90f6-55d3ac1476e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.226556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10dca7-e0da-4738-90f6-55d3ac1476e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:22.226798 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10dca7-e0da-4738-90f6-55d3ac1476e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:22.228261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb10dca7-e0da-4738-90f6-55d3ac1476e6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:22.246031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 410/811] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:22.246226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf1b80-baa5-40cf-a69a-27198b6c9eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:22.246226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf1b80-baa5-40cf-a69a-27198b6c9eea None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.246380 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47577d91-fce0-4b83-8bf4-9b3adea31ee2 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.246490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf1b80-baa5-40cf-a69a-27198b6c9eea None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.246569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47577d91-fce0-4b83-8bf4-9b3adea31ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.246639 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf1b80-baa5-40cf-a69a-27198b6c9eea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.246706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47577d91-fce0-4b83-8bf4-9b3adea31ee2 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.246706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 411/812] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:22.246706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47577d91-fce0-4b83-8bf4-9b3adea31ee2 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:22.256976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf1b80-baa5-40cf-a69a-27198b6c9eea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:22.258038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96bf1b80-baa5-40cf-a69a-27198b6c9eea None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-96bf1b80-baa5-40cf-a69a-27198b6c9eea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:22.275101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:22.276119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:22.277026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:22.277026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:22.277026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:22.277026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:22.277026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:22.277026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:22.277026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:22.277026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:22.277026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 402/813] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:22.277523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf1b80-baa5-40cf-a69a-27198b6c9eea None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:22.311093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:22.363033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47577d91-fce0-4b83-8bf4-9b3adea31ee2 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:22.386590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf1b80-baa5-40cf-a69a-27198b6c9eea None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:22.390276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:22.399741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a69da68-dc24-4370-9814-428c71257328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:22.400374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69da68-dc24-4370-9814-428c71257328 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.400646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69da68-dc24-4370-9814-428c71257328 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.400943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a69da68-dc24-4370-9814-428c71257328 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.401484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69da68-dc24-4370-9814-428c71257328 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:22.402869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a69da68-dc24-4370-9814-428c71257328 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-2a69da68-dc24-4370-9814-428c71257328 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:22.412558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:22.413559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:22.414386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 412/814] 10.222.0.26 () {56 vars in 1129 bytes} [Sat Sep 10 14:01:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:22.415882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4fd5-e459-4b4d-859d-339ac53e9ec9 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.415882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4fd5-e459-4b4d-859d-339ac53e9ec9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.415882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4fd5-e459-4b4d-859d-339ac53e9ec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.433806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121276cb-f9c9-4763-a50a-d6d5c8623841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:22.434809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121276cb-f9c9-4763-a50a-d6d5c8623841 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.435284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121276cb-f9c9-4763-a50a-d6d5c8623841 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.435823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121276cb-f9c9-4763-a50a-d6d5c8623841 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.439422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c0ca4fd5-e459-4b4d-859d-339ac53e9ec9 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:22.440211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c0ca4fd5-e459-4b4d-859d-339ac53e9ec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:22.446004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121276cb-f9c9-4763-a50a-d6d5c8623841 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:22.447567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121276cb-f9c9-4763-a50a-d6d5c8623841 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-121276cb-f9c9-4763-a50a-d6d5c8623841 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:22.461200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:22.462198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:22.464225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 403/815] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 10 14:01:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:22.467634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121276cb-f9c9-4763-a50a-d6d5c8623841 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:22.517578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 413/816] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:22.518532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4fd5-e459-4b4d-859d-339ac53e9ec9 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:22.579845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:22.593537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121276cb-f9c9-4763-a50a-d6d5c8623841 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:22.637269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f6527-47ab-4642-8703-60eefe87a79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:22.638278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f6527-47ab-4642-8703-60eefe87a79f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.638765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f6527-47ab-4642-8703-60eefe87a79f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.639243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f6527-47ab-4642-8703-60eefe87a79f None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.640176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f6527-47ab-4642-8703-60eefe87a79f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:22.640709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f6527-47ab-4642-8703-60eefe87a79f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:22.642397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f6527-47ab-4642-8703-60eefe87a79f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:22.653486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4fd5-e459-4b4d-859d-339ac53e9ec9 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:22.656203 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 404/817] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:22.657363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f6527-47ab-4642-8703-60eefe87a79f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:22.683818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:22.700100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626e022-83bb-491f-a606-ae55707be64e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:22.700709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e022-83bb-491f-a606-ae55707be64e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.700987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e022-83bb-491f-a606-ae55707be64e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.701255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626e022-83bb-491f-a606-ae55707be64e None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.701890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626e022-83bb-491f-a606-ae55707be64e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:22.702193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626e022-83bb-491f-a606-ae55707be64e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:22.704112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626e022-83bb-491f-a606-ae55707be64e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:22.716114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 414/818] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 10 14:01:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:22.716879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626e022-83bb-491f-a606-ae55707be64e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:22.772569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f6527-47ab-4642-8703-60eefe87a79f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:22.819640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46170303-4b72-4a2e-a9fd-2a178325f281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:22.820990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46170303-4b72-4a2e-a9fd-2a178325f281 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.821787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46170303-4b72-4a2e-a9fd-2a178325f281 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.822520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46170303-4b72-4a2e-a9fd-2a178325f281 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.823481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46170303-4b72-4a2e-a9fd-2a178325f281 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:22.824149 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46170303-4b72-4a2e-a9fd-2a178325f281 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:22.826524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46170303-4b72-4a2e-a9fd-2a178325f281 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:22.844466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626e022-83bb-491f-a606-ae55707be64e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:22.851210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 405/819] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 10 14:01:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:22.851210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46170303-4b72-4a2e-a9fd-2a178325f281 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:22.859413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:22.888719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8828f-29ad-4339-bdb4-ff63d2c0a211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:22.889396 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8828f-29ad-4339-bdb4-ff63d2c0a211 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:22.889672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8828f-29ad-4339-bdb4-ff63d2c0a211 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:22.889946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8828f-29ad-4339-bdb4-ff63d2c0a211 None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/d75640e511e64661826c8b8d7aac2ce7/roles/a24499651a7b4225ac5459b8b59f1e24` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:22.907017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8828f-29ad-4339-bdb4-ff63d2c0a211 None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=d75640e511e64661826c8b8d7aac2ce7, role_id=a24499651a7b4225ac5459b8b59f1e24)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:22.908535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8828f-29ad-4339-bdb4-ff63d2c0a211 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:22.931487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 415/820] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:01:22 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/d75640e511e64661826c8b8d7aac2ce7/roles/a24499651a7b4225ac5459b8b59f1e24 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:01:22.931487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8828f-29ad-4339-bdb4-ff63d2c0a211 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:22.976399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:23.061981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8828f-29ad-4339-bdb4-ff63d2c0a211 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:23.102397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46170303-4b72-4a2e-a9fd-2a178325f281 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:23.104298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca3b32-cfa3-4a59-89d2-ba59ae7bb55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:23.104957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca3b32-cfa3-4a59-89d2-ba59ae7bb55b None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:23.105230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca3b32-cfa3-4a59-89d2-ba59ae7bb55b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:23.105498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca3b32-cfa3-4a59-89d2-ba59ae7bb55b None admin] PATH_INFO: `/v3/services/network` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:23.106091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca3b32-cfa3-4a59-89d2-ba59ae7bb55b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:23.107501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca3b32-cfa3-4a59-89d2-ba59ae7bb55b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-d4ca3b32-cfa3-4a59-89d2-ba59ae7bb55b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:23.118220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:23.119459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:23.120611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 416/821] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:22 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:23.124403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca3b32-cfa3-4a59-89d2-ba59ae7bb55b None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:23.153921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cade3-1abb-4c35-aa86-0dc1882b9d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:23.154700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cade3-1abb-4c35-aa86-0dc1882b9d87 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:23.155079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cade3-1abb-4c35-aa86-0dc1882b9d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:23.155494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76cade3-1abb-4c35-aa86-0dc1882b9d87 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:23.156208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cade3-1abb-4c35-aa86-0dc1882b9d87 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:23.156608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cade3-1abb-4c35-aa86-0dc1882b9d87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:23.158117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76cade3-1abb-4c35-aa86-0dc1882b9d87 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:23.165761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 406/822] 10.222.0.26 () {56 vars in 1124 bytes} [Sat Sep 10 14:01:22 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:23.166700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cade3-1abb-4c35-aa86-0dc1882b9d87 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:23.244363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:23.248682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca3b32-cfa3-4a59-89d2-ba59ae7bb55b None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:23.262044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:23.287776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda4250-2f5a-40df-bde4-a6049aa865c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:23.288808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda4250-2f5a-40df-bde4-a6049aa865c9 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:23.289239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda4250-2f5a-40df-bde4-a6049aa865c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:23.289670 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda4250-2f5a-40df-bde4-a6049aa865c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:23.296654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76cade3-1abb-4c35-aa86-0dc1882b9d87 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:23.334965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f2c4d-5de8-4cbe-90a6-8485ff050f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:23.336423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f2c4d-5de8-4cbe-90a6-8485ff050f14 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:23.336701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f2c4d-5de8-4cbe-90a6-8485ff050f14 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:23.336968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f2c4d-5de8-4cbe-90a6-8485ff050f14 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:23.337598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f2c4d-5de8-4cbe-90a6-8485ff050f14 None admin] RBAC: Adding query filter params (name=network) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:23.337896 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f2c4d-5de8-4cbe-90a6-8485ff050f14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:23.339266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f2c4d-5de8-4cbe-90a6-8485ff050f14 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:23.345966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 407/823] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:23.346774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f2c4d-5de8-4cbe-90a6-8485ff050f14 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:23.379306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda4250-2f5a-40df-bde4-a6049aa865c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:23.383203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda4250-2f5a-40df-bde4-a6049aa865c9 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:23.496232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f2c4d-5de8-4cbe-90a6-8485ff050f14 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:23.512891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:23.517633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 417/824] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:23 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:01:23.519047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda4250-2f5a-40df-bde4-a6049aa865c9 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:23.537697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6bed8-4afb-425b-a5cc-5aa28473ede0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:23.538438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6bed8-4afb-425b-a5cc-5aa28473ede0 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:23.538812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6bed8-4afb-425b-a5cc-5aa28473ede0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:23.539208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea6bed8-4afb-425b-a5cc-5aa28473ede0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:23.540104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6bed8-4afb-425b-a5cc-5aa28473ede0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:23.541675 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea6bed8-4afb-425b-a5cc-5aa28473ede0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:23.551410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:23.571522 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 408/825] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:01:23 2022] POST /identity/v3/endpoints => generated 346 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:23.572639 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6bed8-4afb-425b-a5cc-5aa28473ede0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:23.690615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda4250-2f5a-40df-bde4-a6049aa865c9 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:23.749660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febec6e0-594e-4ee6-8697-6abf95b51ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:23.749660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febec6e0-594e-4ee6-8697-6abf95b51ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:23.749660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febec6e0-594e-4ee6-8697-6abf95b51ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:23.749660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febec6e0-594e-4ee6-8697-6abf95b51ad8 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:23.750088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febec6e0-594e-4ee6-8697-6abf95b51ad8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:23.750088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febec6e0-594e-4ee6-8697-6abf95b51ad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:23.750088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febec6e0-594e-4ee6-8697-6abf95b51ad8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:23.756615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 418/826] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:23 2022] GET /identity/v3/services?type=network => generated 350 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:23.762386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febec6e0-594e-4ee6-8697-6abf95b51ad8 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:23.804994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea6bed8-4afb-425b-a5cc-5aa28473ede0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:23.841719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:23.856964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc401e-7a26-4483-b521-3cb9fc52a6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:23.857601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc401e-7a26-4483-b521-3cb9fc52a6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:23.857879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc401e-7a26-4483-b521-3cb9fc52a6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:23.858154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc401e-7a26-4483-b521-3cb9fc52a6a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:23.873498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc401e-7a26-4483-b521-3cb9fc52a6a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:23.876522 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc401e-7a26-4483-b521-3cb9fc52a6a0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-eafc401e-7a26-4483-b521-3cb9fc52a6a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:23.896104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:23.897073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:23.897952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:23.897952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:23.897952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:23.898119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febec6e0-594e-4ee6-8697-6abf95b51ad8 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:23.899520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 409/827] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:23.904427 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc401e-7a26-4483-b521-3cb9fc52a6a0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:23.961273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc05c48-87cb-4fb1-9895-ddd3524d723e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:23.961890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05c48-87cb-4fb1-9895-ddd3524d723e None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:23.962160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05c48-87cb-4fb1-9895-ddd3524d723e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:23.962437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc05c48-87cb-4fb1-9895-ddd3524d723e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:23.963110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc05c48-87cb-4fb1-9895-ddd3524d723e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:23.964531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc05c48-87cb-4fb1-9895-ddd3524d723e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:23.986157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:23.994523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 419/828] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:01:23 2022] POST /identity/v3/endpoints => generated 337 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:24.056333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc401e-7a26-4483-b521-3cb9fc52a6a0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:24.105835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7fa31-f633-4451-807f-80b87adeee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:24.106464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7fa31-f633-4451-807f-80b87adeee5d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:24.106741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7fa31-f633-4451-807f-80b87adeee5d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:24.107016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7fa31-f633-4451-807f-80b87adeee5d None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:24.107684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7fa31-f633-4451-807f-80b87adeee5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:24.108037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7fa31-f633-4451-807f-80b87adeee5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:24.109414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7fa31-f633-4451-807f-80b87adeee5d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:24.128939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 410/829] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:24.134349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc05c48-87cb-4fb1-9895-ddd3524d723e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:24.246923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc05c48-87cb-4fb1-9895-ddd3524d723e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:24.265264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:24.286459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4d4e6-2fef-414f-9cc2-35b6f4880c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:24.287843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4d4e6-2fef-414f-9cc2-35b6f4880c5d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:24.288123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4d4e6-2fef-414f-9cc2-35b6f4880c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:24.288435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4d4e6-2fef-414f-9cc2-35b6f4880c5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:24.295824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4d4e6-2fef-414f-9cc2-35b6f4880c5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:24.303507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4d4e6-2fef-414f-9cc2-35b6f4880c5d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-a6a4d4e6-2fef-414f-9cc2-35b6f4880c5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:24.321635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:24.322664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:24.323561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:24.323561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:24.323561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:24.323561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:24.323561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:24.323561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:24.323561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:24.323561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:24.323561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 420/830] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:24.329198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7fa31-f633-4451-807f-80b87adeee5d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:24.465454 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7fa31-f633-4451-807f-80b87adeee5d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:24.467504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:24.496877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d97ca-5b6d-4b11-8642-5d6f8789d5d5 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:24.497204 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d97ca-5b6d-4b11-8642-5d6f8789d5d5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:24.497431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0d97ca-5b6d-4b11-8642-5d6f8789d5d5 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:24.500088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 421/831] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:24.507400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dcd88-131b-4885-9402-42b71957125d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:24.507714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dcd88-131b-4885-9402-42b71957125d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:24.507989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29dcd88-131b-4885-9402-42b71957125d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:24.509222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ef181-72b8-44be-b116-60d86ec4c172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:24.510349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ef181-72b8-44be-b116-60d86ec4c172 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:24.510629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ef181-72b8-44be-b116-60d86ec4c172 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:24.510902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ef181-72b8-44be-b116-60d86ec4c172 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:24.511633 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ef181-72b8-44be-b116-60d86ec4c172 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:24.511939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ef181-72b8-44be-b116-60d86ec4c172 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:24.513371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ef181-72b8-44be-b116-60d86ec4c172 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:24.528501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 411/832] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:24.543024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ef181-72b8-44be-b116-60d86ec4c172 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:24.546624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e29dcd88-131b-4885-9402-42b71957125d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:24.548461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e29dcd88-131b-4885-9402-42b71957125d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:24.641675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 422/833] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:24.663577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ef181-72b8-44be-b116-60d86ec4c172 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:24.699016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c30f6-ced0-475c-b23b-d4498136345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:24.699646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c30f6-ced0-475c-b23b-d4498136345d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:24.699925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c30f6-ced0-475c-b23b-d4498136345d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:24.700197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c30f6-ced0-475c-b23b-d4498136345d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:24.700922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c30f6-ced0-475c-b23b-d4498136345d None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=cb24a1b55ee7408ab24c68991aec54ec) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:24.701226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c30f6-ced0-475c-b23b-d4498136345d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:24.702662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c30f6-ced0-475c-b23b-d4498136345d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:24.715532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 412/834] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:24 2022] GET /identity/v3/role_assignments?user.id=cb24a1b55ee7408ab24c68991aec54ec&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:24.932704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dcd88-131b-4885-9402-42b71957125d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:25.038642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29dcd88-131b-4885-9402-42b71957125d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:25.047916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:25.077054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b343-116a-4b2d-be62-9e1131b25952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:25.077691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b343-116a-4b2d-be62-9e1131b25952 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:25.077970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b343-116a-4b2d-be62-9e1131b25952 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:25.078237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596b343-116a-4b2d-be62-9e1131b25952 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:25.088878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596b343-116a-4b2d-be62-9e1131b25952 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-e596b343-116a-4b2d-be62-9e1131b25952 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:25.093793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:25.094704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:25.096258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 423/835] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:25.101853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c30f6-ced0-475c-b23b-d4498136345d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:25.132054 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:25.209287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c30f6-ced0-475c-b23b-d4498136345d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:25.244265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae39d10-c95b-4dbb-93ef-553c734c6ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:25.244867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae39d10-c95b-4dbb-93ef-553c734c6ebb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:25.245142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae39d10-c95b-4dbb-93ef-553c734c6ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:25.245414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae39d10-c95b-4dbb-93ef-553c734c6ebb None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:25.246078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae39d10-c95b-4dbb-93ef-553c734c6ebb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:25.246375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae39d10-c95b-4dbb-93ef-553c734c6ebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:25.247777 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae39d10-c95b-4dbb-93ef-553c734c6ebb None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:25.257337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 413/836] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:01:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:25.263042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b343-116a-4b2d-be62-9e1131b25952 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:25.382237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596b343-116a-4b2d-be62-9e1131b25952 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:25.417884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae83181-4010-41e1-a2fa-e5839ac7c382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:25.418929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae83181-4010-41e1-a2fa-e5839ac7c382 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:25.419469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae83181-4010-41e1-a2fa-e5839ac7c382 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:25.420000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae83181-4010-41e1-a2fa-e5839ac7c382 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:25.425225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa77c7-e615-4d98-b2db-fce91cdea786 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:25.426050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa77c7-e615-4d98-b2db-fce91cdea786 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:25.426314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3aa77c7-e615-4d98-b2db-fce91cdea786 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:25.427969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 414/837] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:25.432780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae83181-4010-41e1-a2fa-e5839ac7c382 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:25.433296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc331-acd0-4824-8637-aa832537a222 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:25.433585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc331-acd0-4824-8637-aa832537a222 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:25.433857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfc331-acd0-4824-8637-aa832537a222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:25.435361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae83181-4010-41e1-a2fa-e5839ac7c382 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-0ae83181-4010-41e1-a2fa-e5839ac7c382 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:25.451706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:25.452717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:25.453588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:25.453588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:25.453588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:25.453588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 424/838] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:25.455383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-98cfc331-acd0-4824-8637-aa832537a222 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:25.457191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-98cfc331-acd0-4824-8637-aa832537a222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:25.458774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae83181-4010-41e1-a2fa-e5839ac7c382 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:25.485854 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:25.542469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 415/839] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:25 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:25.579886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae83181-4010-41e1-a2fa-e5839ac7c382 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:25.618130 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a5238-201a-467a-aab0-7e3139937832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:25.618993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a5238-201a-467a-aab0-7e3139937832 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:25.619425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a5238-201a-467a-aab0-7e3139937832 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:25.619876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1a5238-201a-467a-aab0-7e3139937832 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:25.620693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a5238-201a-467a-aab0-7e3139937832 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:25.621165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a5238-201a-467a-aab0-7e3139937832 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:25.622735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1a5238-201a-467a-aab0-7e3139937832 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:25.636514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 425/840] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:25.644627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfc331-acd0-4824-8637-aa832537a222 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:25.662559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:25.749558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfc331-acd0-4824-8637-aa832537a222 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:25.787093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbd7e6-264e-4f16-9a67-555ec44228dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:25.787749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbd7e6-264e-4f16-9a67-555ec44228dc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:25.788031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbd7e6-264e-4f16-9a67-555ec44228dc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:25.788300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbd7e6-264e-4f16-9a67-555ec44228dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:25.796260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbd7e6-264e-4f16-9a67-555ec44228dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:25.797763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbd7e6-264e-4f16-9a67-555ec44228dc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-94dbd7e6-264e-4f16-9a67-555ec44228dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:25.811254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:25.812330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:25.814288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 416/841] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:25.822252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a5238-201a-467a-aab0-7e3139937832 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:25.833436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbd7e6-264e-4f16-9a67-555ec44228dc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:25.915762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:25.935392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:25.945530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1a5238-201a-467a-aab0-7e3139937832 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:25.963030 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbd7e6-264e-4f16-9a67-555ec44228dc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:25.989310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6bb32-1467-4469-818f-140f09bfa46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:25.990202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6bb32-1467-4469-818f-140f09bfa46c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:25.990624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6bb32-1467-4469-818f-140f09bfa46c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:25.991050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a6bb32-1467-4469-818f-140f09bfa46c None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:25.992055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6bb32-1467-4469-818f-140f09bfa46c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:25.992814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6bb32-1467-4469-818f-140f09bfa46c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:25.995432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a6bb32-1467-4469-818f-140f09bfa46c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:26.006021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6a4f0-d25a-4e2b-ab0f-fe46e7d6325b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:26.006814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6a4f0-d25a-4e2b-ab0f-fe46e7d6325b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.007090 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6a4f0-d25a-4e2b-ab0f-fe46e7d6325b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.007508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6a4f0-d25a-4e2b-ab0f-fe46e7d6325b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.018216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 426/842] 10.222.0.26 () {56 vars in 1135 bytes} [Sat Sep 10 14:01:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:26.019806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6a4f0-d25a-4e2b-ab0f-fe46e7d6325b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:26.024047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6bb32-1467-4469-818f-140f09bfa46c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-f9b6a4f0-d25a-4e2b-ab0f-fe46e7d6325b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:26.025574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:26.026488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:26.027375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 417/843] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:01:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:26.032680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6a4f0-d25a-4e2b-ab0f-fe46e7d6325b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:26.140694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6a4f0-d25a-4e2b-ab0f-fe46e7d6325b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:26.141047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a6bb32-1467-4469-818f-140f09bfa46c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:26.181095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329f96a-96ed-442d-9cf6-16c6da495ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:26.181703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329f96a-96ed-442d-9cf6-16c6da495ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.181956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e6793-0a57-43c0-bed3-ee0f2713626b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:26.182098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329f96a-96ed-442d-9cf6-16c6da495ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.182275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329f96a-96ed-442d-9cf6-16c6da495ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.182879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329f96a-96ed-442d-9cf6-16c6da495ee9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:26.183175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329f96a-96ed-442d-9cf6-16c6da495ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:26.183512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e6793-0a57-43c0-bed3-ee0f2713626b None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.183990 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e6793-0a57-43c0-bed3-ee0f2713626b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.184484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e6793-0a57-43c0-bed3-ee0f2713626b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.184587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329f96a-96ed-442d-9cf6-16c6da495ee9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:26.188411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:26.190466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:26.199679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 418/844] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:01:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:26.201757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e6f7-f475-4246-9147-ca9e9c2592ef None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.202803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e6f7-f475-4246-9147-ca9e9c2592ef None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.203076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511e6f7-f475-4246-9147-ca9e9c2592ef None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.206151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 419/845] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:26.206959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511e6f7-f475-4246-9147-ca9e9c2592ef None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:26.262435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e6793-0a57-43c0-bed3-ee0f2713626b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:26.263828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e6793-0a57-43c0-bed3-ee0f2713626b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:26.313862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511e6f7-f475-4246-9147-ca9e9c2592ef None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:26.349224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7e66f-00a6-457e-ba90-865ee67bfb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:26.349839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e66f-00a6-457e-ba90-865ee67bfb5f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.350113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e66f-00a6-457e-ba90-865ee67bfb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.350383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c7e66f-00a6-457e-ba90-865ee67bfb5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.360935 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7e66f-00a6-457e-ba90-865ee67bfb5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:26.363623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c7e66f-00a6-457e-ba90-865ee67bfb5f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:26.371018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 427/846] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:26 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:01:26.372297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb853a-3560-422d-97eb-f0b6615799a2 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.372576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb853a-3560-422d-97eb-f0b6615799a2 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.372847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb853a-3560-422d-97eb-f0b6615799a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-30c7e66f-00a6-457e-ba90-865ee67bfb5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:26.382425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:26.383390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:26.384296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:26.384296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:26.384296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:26.384296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 420/847] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:26.385043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7e66f-00a6-457e-ba90-865ee67bfb5f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:26.396049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-03fb853a-3560-422d-97eb-f0b6615799a2 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:26.397237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-03fb853a-3560-422d-97eb-f0b6615799a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:26.432642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:26.440658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:26.478917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 428/848] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:26.480247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb853a-3560-422d-97eb-f0b6615799a2 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:26.500938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c7e66f-00a6-457e-ba90-865ee67bfb5f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:26.539592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f18d5e-28f3-4ba8-b5e8-9f43ecbfcc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:26.540620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f18d5e-28f3-4ba8-b5e8-9f43ecbfcc01 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.541078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f18d5e-28f3-4ba8-b5e8-9f43ecbfcc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.541525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f18d5e-28f3-4ba8-b5e8-9f43ecbfcc01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.554067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f18d5e-28f3-4ba8-b5e8-9f43ecbfcc01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:26.556410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f18d5e-28f3-4ba8-b5e8-9f43ecbfcc01 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-50f18d5e-28f3-4ba8-b5e8-9f43ecbfcc01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:26.572170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:26.573169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:26.574033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:26.574033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:26.574033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:26.574033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 421/849] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:26.580365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f18d5e-28f3-4ba8-b5e8-9f43ecbfcc01 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:26.616176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb853a-3560-422d-97eb-f0b6615799a2 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:26.666557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4921c9-4540-4df5-9e8c-f57b344c438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:26.667931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4921c9-4540-4df5-9e8c-f57b344c438e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.668752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4921c9-4540-4df5-9e8c-f57b344c438e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.669195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4921c9-4540-4df5-9e8c-f57b344c438e None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.670249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4921c9-4540-4df5-9e8c-f57b344c438e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:26.670697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4921c9-4540-4df5-9e8c-f57b344c438e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:26.673129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4921c9-4540-4df5-9e8c-f57b344c438e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:26.690593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 429/850] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:26.692941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f18d5e-28f3-4ba8-b5e8-9f43ecbfcc01 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:26.696152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4921c9-4540-4df5-9e8c-f57b344c438e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:26.696979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:26.727800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:26.737297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae35db2-ffbc-499a-bb27-8e9a6a2d5fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:26.738045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae35db2-ffbc-499a-bb27-8e9a6a2d5fb8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.739716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae35db2-ffbc-499a-bb27-8e9a6a2d5fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.739997 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae35db2-ffbc-499a-bb27-8e9a6a2d5fb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.740653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae35db2-ffbc-499a-bb27-8e9a6a2d5fb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:26.740961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae35db2-ffbc-499a-bb27-8e9a6a2d5fb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:26.742358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae35db2-ffbc-499a-bb27-8e9a6a2d5fb8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:26.761100 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 422/851] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:26.762406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae35db2-ffbc-499a-bb27-8e9a6a2d5fb8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:26.815504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4921c9-4540-4df5-9e8c-f57b344c438e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:26.854453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9ba52-e470-4393-a7b4-9dfe6947636e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:26.855401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9ba52-e470-4393-a7b4-9dfe6947636e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.855801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9ba52-e470-4393-a7b4-9dfe6947636e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.856002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9ba52-e470-4393-a7b4-9dfe6947636e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.864593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9ba52-e470-4393-a7b4-9dfe6947636e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:26.866055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f9ba52-e470-4393-a7b4-9dfe6947636e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-c5f9ba52-e470-4393-a7b4-9dfe6947636e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:26.877343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:26.878276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 430/852] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 10 14:01:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:26.880180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9ba52-e470-4393-a7b4-9dfe6947636e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:26.882349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae35db2-ffbc-499a-bb27-8e9a6a2d5fb8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:26.918202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fff98-b637-4753-8c33-843d8f4be254 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:26.918823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fff98-b637-4753-8c33-843d8f4be254 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:26.919096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fff98-b637-4753-8c33-843d8f4be254 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:26.919363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510fff98-b637-4753-8c33-843d8f4be254 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:26.927167 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fff98-b637-4753-8c33-843d8f4be254 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:26.928646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510fff98-b637-4753-8c33-843d8f4be254 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-510fff98-b637-4753-8c33-843d8f4be254 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:26.941581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:26.943058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:26.944629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:26.944629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:26.944629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:26.944629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:26.944629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:26.944629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:26.944629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:26.944629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:26.944629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 423/853] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:26.944629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fff98-b637-4753-8c33-843d8f4be254 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:26.970610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:26.986795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:27.003498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f9ba52-e470-4393-a7b4-9dfe6947636e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:27.040659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda101f3-6df8-423d-b015-97c7a28d2ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:27.041464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda101f3-6df8-423d-b015-97c7a28d2ecb None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.041601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda101f3-6df8-423d-b015-97c7a28d2ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.041852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda101f3-6df8-423d-b015-97c7a28d2ecb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.042442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda101f3-6df8-423d-b015-97c7a28d2ecb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:27.043840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda101f3-6df8-423d-b015-97c7a28d2ecb None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-cda101f3-6df8-423d-b015-97c7a28d2ecb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:27.053550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:27.054579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:27.055431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 431/854] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:01:26 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:27.056303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda101f3-6df8-423d-b015-97c7a28d2ecb None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:27.064015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510fff98-b637-4753-8c33-843d8f4be254 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:27.099652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743774c-1bc5-4523-9e59-df3ceae00d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:27.100172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743774c-1bc5-4523-9e59-df3ceae00d1e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.100447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743774c-1bc5-4523-9e59-df3ceae00d1e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.100713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743774c-1bc5-4523-9e59-df3ceae00d1e None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.101430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743774c-1bc5-4523-9e59-df3ceae00d1e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:27.101730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743774c-1bc5-4523-9e59-df3ceae00d1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:27.103154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1743774c-1bc5-4523-9e59-df3ceae00d1e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:27.122121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 424/855] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 10 14:01:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:27.127372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743774c-1bc5-4523-9e59-df3ceae00d1e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:27.161779 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda101f3-6df8-423d-b015-97c7a28d2ecb None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:27.194481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa735063-40f7-4d7d-9cbe-41957442cf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:27.195077 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa735063-40f7-4d7d-9cbe-41957442cf39 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.195351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa735063-40f7-4d7d-9cbe-41957442cf39 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.195650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa735063-40f7-4d7d-9cbe-41957442cf39 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.196337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa735063-40f7-4d7d-9cbe-41957442cf39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:27.196640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa735063-40f7-4d7d-9cbe-41957442cf39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:27.198064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa735063-40f7-4d7d-9cbe-41957442cf39 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:27.208010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:27.210512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 432/856] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:27.211357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa735063-40f7-4d7d-9cbe-41957442cf39 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:27.227261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:27.229552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1743774c-1bc5-4523-9e59-df3ceae00d1e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:27.264176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b325a-89f5-4df7-845c-e21ba4f2579b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:27.264470 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b325a-89f5-4df7-845c-e21ba4f2579b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.264747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b325a-89f5-4df7-845c-e21ba4f2579b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.265023 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b325a-89f5-4df7-845c-e21ba4f2579b None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.265648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b325a-89f5-4df7-845c-e21ba4f2579b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:27.265947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b325a-89f5-4df7-845c-e21ba4f2579b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:27.267324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b325a-89f5-4df7-845c-e21ba4f2579b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:27.272398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 425/857] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:01:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:27.273527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc318d97-86a6-47ec-9e32-791ce19fb901 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.273805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc318d97-86a6-47ec-9e32-791ce19fb901 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.274074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc318d97-86a6-47ec-9e32-791ce19fb901 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.275269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 426/858] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:27.275986 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc318d97-86a6-47ec-9e32-791ce19fb901 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:27.321068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa735063-40f7-4d7d-9cbe-41957442cf39 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:27.354837 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf72ca-daca-42b1-b86b-00013619c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:27.355364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf72ca-daca-42b1-b86b-00013619c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.355669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf72ca-daca-42b1-b86b-00013619c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.355960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf72ca-daca-42b1-b86b-00013619c9a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.375711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc318d97-86a6-47ec-9e32-791ce19fb901 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:27.408906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62802518-d793-42c8-8d9d-52837cf9a0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:27.409497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62802518-d793-42c8-8d9d-52837cf9a0fd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.410365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62802518-d793-42c8-8d9d-52837cf9a0fd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.410657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62802518-d793-42c8-8d9d-52837cf9a0fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.411374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62802518-d793-42c8-8d9d-52837cf9a0fd None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=bc0a7e1b772b4703ac2e5ec33b5ecf6b) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:27.411703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62802518-d793-42c8-8d9d-52837cf9a0fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:27.413126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62802518-d793-42c8-8d9d-52837cf9a0fd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:27.418761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 427/859] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:27 2022] GET /identity/v3/role_assignments?user.id=bc0a7e1b772b4703ac2e5ec33b5ecf6b&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:27.419552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62802518-d793-42c8-8d9d-52837cf9a0fd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:27.423182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf72ca-daca-42b1-b86b-00013619c9a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:27.424708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf72ca-daca-42b1-b86b-00013619c9a1 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:27.439082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:27.454561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:27.550551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 433/860] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:01:27.552504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec10b6-f3d9-4327-b16f-b40438da8856 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.552788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec10b6-f3d9-4327-b16f-b40438da8856 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.553061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec10b6-f3d9-4327-b16f-b40438da8856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.562405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62802518-d793-42c8-8d9d-52837cf9a0fd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:27.572092 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7aec10b6-f3d9-4327-b16f-b40438da8856 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:27.573315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7aec10b6-f3d9-4327-b16f-b40438da8856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:27.606714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daec73-538c-47f7-aedd-e8cf757745f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:27.607239 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daec73-538c-47f7-aedd-e8cf757745f4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.607538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daec73-538c-47f7-aedd-e8cf757745f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.607804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6daec73-538c-47f7-aedd-e8cf757745f4 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.608427 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6daec73-538c-47f7-aedd-e8cf757745f4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:27.608719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6daec73-538c-47f7-aedd-e8cf757745f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:27.610088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6daec73-538c-47f7-aedd-e8cf757745f4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:27.616338 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 428/861] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:01:27 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:27.617067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daec73-538c-47f7-aedd-e8cf757745f4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:27.660679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 434/862] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:27 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:27.722142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:27.751884 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6daec73-538c-47f7-aedd-e8cf757745f4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:27.792249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574397b3-f0ac-4f7f-8739-ee7c39409a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:27.792890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574397b3-f0ac-4f7f-8739-ee7c39409a79 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.793162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574397b3-f0ac-4f7f-8739-ee7c39409a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.793429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574397b3-f0ac-4f7f-8739-ee7c39409a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.805255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574397b3-f0ac-4f7f-8739-ee7c39409a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:27.806712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574397b3-f0ac-4f7f-8739-ee7c39409a79 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-574397b3-f0ac-4f7f-8739-ee7c39409a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:27.821934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:27.822943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:27.823924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:27.823924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:27.823924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:27.827602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 429/863] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:27.828961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec10b6-f3d9-4327-b16f-b40438da8856 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:27.875574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:27.936847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec10b6-f3d9-4327-b16f-b40438da8856 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:27.963414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574397b3-f0ac-4f7f-8739-ee7c39409a79 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:27.976913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a0f9e-97e4-437d-b3bc-78fd9aded265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:27.977862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a0f9e-97e4-437d-b3bc-78fd9aded265 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:27.978491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a0f9e-97e4-437d-b3bc-78fd9aded265 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:27.978784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a0f9e-97e4-437d-b3bc-78fd9aded265 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:27.979474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a0f9e-97e4-437d-b3bc-78fd9aded265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:27.979769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a0f9e-97e4-437d-b3bc-78fd9aded265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:27.981146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a0f9e-97e4-437d-b3bc-78fd9aded265 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:27.998733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 435/864] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:28.001596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a0f9e-97e4-437d-b3bc-78fd9aded265 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:28.091225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574397b3-f0ac-4f7f-8739-ee7c39409a79 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:28.118746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:28.122784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a0f9e-97e4-437d-b3bc-78fd9aded265 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:28.123414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:28.136047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746fdf0-d3e3-4b0a-b8cc-8fc9b6970346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:28.137289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fdf0-d3e3-4b0a-b8cc-8fc9b6970346 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:28.137573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fdf0-d3e3-4b0a-b8cc-8fc9b6970346 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:28.137845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b746fdf0-d3e3-4b0a-b8cc-8fc9b6970346 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:28.138444 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746fdf0-d3e3-4b0a-b8cc-8fc9b6970346 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:28.140414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b746fdf0-d3e3-4b0a-b8cc-8fc9b6970346 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-b746fdf0-d3e3-4b0a-b8cc-8fc9b6970346 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:28.152807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:28.153990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 14:01:28.155084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:28.158971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 430/865] 10.222.0.26 () {56 vars in 1132 bytes} [Sat Sep 10 14:01:27 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:28.161061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746fdf0-d3e3-4b0a-b8cc-8fc9b6970346 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:28.172174 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679169cf-a106-4cd0-9985-f26c4c21af5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:28.172810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679169cf-a106-4cd0-9985-f26c4c21af5a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:28.173085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679169cf-a106-4cd0-9985-f26c4c21af5a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:28.173351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679169cf-a106-4cd0-9985-f26c4c21af5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:28.182276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679169cf-a106-4cd0-9985-f26c4c21af5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:28.184035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679169cf-a106-4cd0-9985-f26c4c21af5a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-679169cf-a106-4cd0-9985-f26c4c21af5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:28.197933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:28.199088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:28.200199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:28.200199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:28.200199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:28.200199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:28.200199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:28.200199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:28.200199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:28.200199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:28.200199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 436/866] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:28.205271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679169cf-a106-4cd0-9985-f26c4c21af5a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:28.281337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b746fdf0-d3e3-4b0a-b8cc-8fc9b6970346 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:28.318785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f4892-f53a-46be-bdd5-2fb92386d2bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:28.319394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f4892-f53a-46be-bdd5-2fb92386d2bd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:28.319849 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f4892-f53a-46be-bdd5-2fb92386d2bd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:28.320123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f4892-f53a-46be-bdd5-2fb92386d2bd None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:28.320811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f4892-f53a-46be-bdd5-2fb92386d2bd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:28.321110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f4892-f53a-46be-bdd5-2fb92386d2bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:28.322630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f4892-f53a-46be-bdd5-2fb92386d2bd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:28.331656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 431/867] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:28.333705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f4892-f53a-46be-bdd5-2fb92386d2bd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:28.341232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679169cf-a106-4cd0-9985-f26c4c21af5a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:28.382052 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69eb73-fd6b-47f2-ae51-ef170e3cc6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:28.382682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69eb73-fd6b-47f2-ae51-ef170e3cc6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:28.382956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69eb73-fd6b-47f2-ae51-ef170e3cc6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:28.383225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69eb73-fd6b-47f2-ae51-ef170e3cc6e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:28.384442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69eb73-fd6b-47f2-ae51-ef170e3cc6e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:28.384956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69eb73-fd6b-47f2-ae51-ef170e3cc6e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:28.387856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69eb73-fd6b-47f2-ae51-ef170e3cc6e0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:28.403801 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:28.409624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 437/868] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:28.412083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69eb73-fd6b-47f2-ae51-ef170e3cc6e0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:28.436032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:28.465457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f4892-f53a-46be-bdd5-2fb92386d2bd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:28.502579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b961c32-9211-4a35-b746-5601aef925b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:28.503736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b961c32-9211-4a35-b746-5601aef925b5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:28.504251 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b961c32-9211-4a35-b746-5601aef925b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:28.504668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b961c32-9211-4a35-b746-5601aef925b5 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:28.505462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b961c32-9211-4a35-b746-5601aef925b5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:28.506006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b961c32-9211-4a35-b746-5601aef925b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:28.507681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b961c32-9211-4a35-b746-5601aef925b5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:28.519880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 432/869] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:28 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:28.523162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69eb73-fd6b-47f2-ae51-ef170e3cc6e0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:28.563245 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f4f511-3045-45b7-88dd-f34b6d805904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:28.563896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f511-3045-45b7-88dd-f34b6d805904 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:28.564169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f511-3045-45b7-88dd-f34b6d805904 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:28.564439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f4f511-3045-45b7-88dd-f34b6d805904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:28.565174 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4f511-3045-45b7-88dd-f34b6d805904 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=d75640e511e64661826c8b8d7aac2ce7) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:28.565476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4f511-3045-45b7-88dd-f34b6d805904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:28.568405 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f4f511-3045-45b7-88dd-f34b6d805904 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:28.583110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 438/870] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:28 2022] GET /identity/v3/role_assignments?user.id=d75640e511e64661826c8b8d7aac2ce7&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:29.696181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae4caf-d8c8-4853-9e49-d1e6958f24a1 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:29.696698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae4caf-d8c8-4853-9e49-d1e6958f24a1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:29.696698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ae4caf-d8c8-4853-9e49-d1e6958f24a1 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:29.698201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 433/871] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:29.703712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc6d5-5e9e-456c-9ce2-ef5d9f82547e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:29.704105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc6d5-5e9e-456c-9ce2-ef5d9f82547e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:29.704286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cc6d5-5e9e-456c-9ce2-ef5d9f82547e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:29.723676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-691cc6d5-5e9e-456c-9ce2-ef5d9f82547e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:29.724851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-691cc6d5-5e9e-456c-9ce2-ef5d9f82547e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:29.800680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 439/872] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:29.876870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716405e5-bdff-4761-b83b-25f7463eab1b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:29.877177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716405e5-bdff-4761-b83b-25f7463eab1b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:29.877425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716405e5-bdff-4761-b83b-25f7463eab1b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:29.879083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 434/873] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:29.884233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc296c-bf66-4a3f-8282-5ada3f469c46 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:29.884520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc296c-bf66-4a3f-8282-5ada3f469c46 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:29.884794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc296c-bf66-4a3f-8282-5ada3f469c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:29.903405 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-57dc296c-bf66-4a3f-8282-5ada3f469c46 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:29.904630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-57dc296c-bf66-4a3f-8282-5ada3f469c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:29.914153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:29.979655 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 440/874] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:30.073156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716405e5-bdff-4761-b83b-25f7463eab1b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:30.180905 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716405e5-bdff-4761-b83b-25f7463eab1b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:30.198412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:30.216887 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffa09b-ec46-48e7-aca7-d75c4cec8e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:30.217523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffa09b-ec46-48e7-aca7-d75c4cec8e4b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:30.217800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffa09b-ec46-48e7-aca7-d75c4cec8e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:30.218073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ffa09b-ec46-48e7-aca7-d75c4cec8e4b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:30.218732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffa09b-ec46-48e7-aca7-d75c4cec8e4b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:30.220483 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ffa09b-ec46-48e7-aca7-d75c4cec8e4b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c4ffa09b-ec46-48e7-aca7-d75c4cec8e4b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:30.229734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:30.230692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:30.231551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 435/875] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:30.237131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc296c-bf66-4a3f-8282-5ada3f469c46 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:30.255069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffa09b-ec46-48e7-aca7-d75c4cec8e4b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:30.360379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc296c-bf66-4a3f-8282-5ada3f469c46 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:30.376096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ffa09b-ec46-48e7-aca7-d75c4cec8e4b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:30.404306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9676-6e78-4a07-8e45-df75bd3e55cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:30.404947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9676-6e78-4a07-8e45-df75bd3e55cb None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:30.405385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9676-6e78-4a07-8e45-df75bd3e55cb None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:30.405657 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9676-6e78-4a07-8e45-df75bd3e55cb None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:30.406617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9676-6e78-4a07-8e45-df75bd3e55cb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:30.407058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9676-6e78-4a07-8e45-df75bd3e55cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:30.408737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9676-6e78-4a07-8e45-df75bd3e55cb None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:30.414891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 441/876] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 10 14:01:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:30.419043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4da97e-2d17-4c54-b71c-9fc3bf43f98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:30.419989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4da97e-2d17-4c54-b71c-9fc3bf43f98b None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:30.420453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9676-6e78-4a07-8e45-df75bd3e55cb None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:30.420702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4da97e-2d17-4c54-b71c-9fc3bf43f98b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:30.421170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4da97e-2d17-4c54-b71c-9fc3bf43f98b None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:30.422059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4da97e-2d17-4c54-b71c-9fc3bf43f98b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:30.425037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4da97e-2d17-4c54-b71c-9fc3bf43f98b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:30.436341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:30.451516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 436/877] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:01:30 2022] POST /identity/v3/services => generated 251 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:30.561967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9676-6e78-4a07-8e45-df75bd3e55cb None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:30.611951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff3a7e-e956-4ee1-ad7d-050c36963448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:30.611951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3a7e-e956-4ee1-ad7d-050c36963448 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:30.611951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3a7e-e956-4ee1-ad7d-050c36963448 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:30.611951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff3a7e-e956-4ee1-ad7d-050c36963448 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:30.612385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff3a7e-e956-4ee1-ad7d-050c36963448 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:30.612385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff3a7e-e956-4ee1-ad7d-050c36963448 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:30.612521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff3a7e-e956-4ee1-ad7d-050c36963448 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:30.618779 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 442/878] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 10 14:01:30 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:30.809643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80557caa-9b1a-4427-a2ac-1c9c90f35dd3 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:30.810187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80557caa-9b1a-4427-a2ac-1c9c90f35dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:30.810187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80557caa-9b1a-4427-a2ac-1c9c90f35dd3 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:30.811930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 437/879] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:30.818030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8288ae2-7c03-43b2-8e1c-fb345b1f380f None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:30.818551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8288ae2-7c03-43b2-8e1c-fb345b1f380f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:30.818551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8288ae2-7c03-43b2-8e1c-fb345b1f380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:30.841220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d8288ae2-7c03-43b2-8e1c-fb345b1f380f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:30.842781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d8288ae2-7c03-43b2-8e1c-fb345b1f380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:30.910820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:30.917142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 443/880] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:30.926846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53f4f7-5565-48e7-b5fa-105e44c6ab7c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:30.927129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53f4f7-5565-48e7-b5fa-105e44c6ab7c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:30.927399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab53f4f7-5565-48e7-b5fa-105e44c6ab7c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:30.929005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 438/881] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:30.934170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9421e-6aef-4b4a-947b-717dbe428522 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:30.934451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9421e-6aef-4b4a-947b-717dbe428522 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:30.934724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9421e-6aef-4b4a-947b-717dbe428522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:30.953593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c9e9421e-6aef-4b4a-947b-717dbe428522 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:30.954767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c9e9421e-6aef-4b4a-947b-717dbe428522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:31.026406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 444/882] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:30 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:31.213872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53f4f7-5565-48e7-b5fa-105e44c6ab7c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:31.260431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:31.308873 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9421e-6aef-4b4a-947b-717dbe428522 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:31.325101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab53f4f7-5565-48e7-b5fa-105e44c6ab7c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:31.361726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2a4d2-2da7-45ce-958c-6973fbc52df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:31.362347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2a4d2-2da7-45ce-958c-6973fbc52df5 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:31.362625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2a4d2-2da7-45ce-958c-6973fbc52df5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:31.363474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2a4d2-2da7-45ce-958c-6973fbc52df5 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:31.364145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2a4d2-2da7-45ce-958c-6973fbc52df5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:31.365638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2a4d2-2da7-45ce-958c-6973fbc52df5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:31.384541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 439/883] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:01:31 2022] POST /identity/v3/services => generated 269 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:31.433750 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e9421e-6aef-4b4a-947b-717dbe428522 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:31.466179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:31.483603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f38d9-8134-478c-a2a3-fe993724c8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:31.484256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f38d9-8134-478c-a2a3-fe993724c8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:31.484535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f38d9-8134-478c-a2a3-fe993724c8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:31.484811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f38d9-8134-478c-a2a3-fe993724c8e9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:31.497560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f38d9-8134-478c-a2a3-fe993724c8e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-860f38d9-8134-478c-a2a3-fe993724c8e9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:31.502656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:31.503721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:31.504794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 445/884] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 10 14:01:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:31.510284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2a4d2-2da7-45ce-958c-6973fbc52df5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:31.628120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2a4d2-2da7-45ce-958c-6973fbc52df5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:31.629880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:31.668458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8e1d8-79b1-4352-94dc-5b5a28cdcfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:31.669426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8e1d8-79b1-4352-94dc-5b5a28cdcfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:31.669906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8e1d8-79b1-4352-94dc-5b5a28cdcfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:31.670394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8e1d8-79b1-4352-94dc-5b5a28cdcfa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:31.671256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8e1d8-79b1-4352-94dc-5b5a28cdcfa4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:31.671808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8e1d8-79b1-4352-94dc-5b5a28cdcfa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:31.673413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8e1d8-79b1-4352-94dc-5b5a28cdcfa4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:31.683165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 440/885] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 10 14:01:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:31.691554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f38d9-8134-478c-a2a3-fe993724c8e9 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:31.801857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f38d9-8134-478c-a2a3-fe993724c8e9 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:31.838948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3661c3-c9cc-4b62-b25b-de2b14e18b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:31.839606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3661c3-c9cc-4b62-b25b-de2b14e18b46 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:31.839877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3661c3-c9cc-4b62-b25b-de2b14e18b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:31.840149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3661c3-c9cc-4b62-b25b-de2b14e18b46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:31.852119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3661c3-c9cc-4b62-b25b-de2b14e18b46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:31.853572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3661c3-c9cc-4b62-b25b-de2b14e18b46 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-bf3661c3-c9cc-4b62-b25b-de2b14e18b46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:31.868604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:31.869549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:31.870487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:31.870487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:31.870487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:31.870487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 446/886] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:31.875371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8e1d8-79b1-4352-94dc-5b5a28cdcfa4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:31.987153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8e1d8-79b1-4352-94dc-5b5a28cdcfa4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:32.022772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bf8f9-2b45-4742-b84f-bafc9844227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:32.023728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bf8f9-2b45-4742-b84f-bafc9844227e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:32.024202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bf8f9-2b45-4742-b84f-bafc9844227e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:32.024648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bf8f9-2b45-4742-b84f-bafc9844227e None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:32.025463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bf8f9-2b45-4742-b84f-bafc9844227e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:32.025939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bf8f9-2b45-4742-b84f-bafc9844227e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:32.027506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bf8f9-2b45-4742-b84f-bafc9844227e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:32.040489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 441/887] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:32.051495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3661c3-c9cc-4b62-b25b-de2b14e18b46 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:32.083663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:32.154465 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3661c3-c9cc-4b62-b25b-de2b14e18b46 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:32.189236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676dbbde-39e6-45a5-b02b-d488695e7390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:32.189870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dbbde-39e6-45a5-b02b-d488695e7390 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:32.190147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dbbde-39e6-45a5-b02b-d488695e7390 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:32.190418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676dbbde-39e6-45a5-b02b-d488695e7390 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:32.199148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676dbbde-39e6-45a5-b02b-d488695e7390 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:32.200634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676dbbde-39e6-45a5-b02b-d488695e7390 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-676dbbde-39e6-45a5-b02b-d488695e7390 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:32.212303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:32.213270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:32.214229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 447/888] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 10 14:01:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:32.219859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bf8f9-2b45-4742-b84f-bafc9844227e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:32.247295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:32.326581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bf8f9-2b45-4742-b84f-bafc9844227e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:32.363775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cc5ce-28ec-4da6-893e-0f0f8a997b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:32.364302 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cc5ce-28ec-4da6-893e-0f0f8a997b84 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:32.364578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cc5ce-28ec-4da6-893e-0f0f8a997b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:32.364850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63cc5ce-28ec-4da6-893e-0f0f8a997b84 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:32.365650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cc5ce-28ec-4da6-893e-0f0f8a997b84 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:32.365956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cc5ce-28ec-4da6-893e-0f0f8a997b84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:32.367545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63cc5ce-28ec-4da6-893e-0f0f8a997b84 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:32.394667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 442/889] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 10 14:01:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:32.400441 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676dbbde-39e6-45a5-b02b-d488695e7390 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:32.519290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:32.530565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676dbbde-39e6-45a5-b02b-d488695e7390 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:32.583352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a526806-0ec3-459b-b25c-71d4b20751b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:32.583999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a526806-0ec3-459b-b25c-71d4b20751b5 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:32.584276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a526806-0ec3-459b-b25c-71d4b20751b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:32.584549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a526806-0ec3-459b-b25c-71d4b20751b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:32.671539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a526806-0ec3-459b-b25c-71d4b20751b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:32.673608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a526806-0ec3-459b-b25c-71d4b20751b5 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:32.792243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:32.796838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 448/890] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:32 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:01:32.803467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cc5ce-28ec-4da6-893e-0f0f8a997b84 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:32.906465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:32.912569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb7433-f562-4789-9bbb-c2297a732edf None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:32.913059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb7433-f562-4789-9bbb-c2297a732edf None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:32.913150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb7433-f562-4789-9bbb-c2297a732edf None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:32.915802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 449/891] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:32.925197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d6b0-9af6-4331-bea7-b3208da7b888 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:32.926275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d6b0-9af6-4331-bea7-b3208da7b888 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:32.926770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b9d6b0-9af6-4331-bea7-b3208da7b888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:32.931191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63cc5ce-28ec-4da6-893e-0f0f8a997b84 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:32.950141 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-39b9d6b0-9af6-4331-bea7-b3208da7b888 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:32.951327 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-39b9d6b0-9af6-4331-bea7-b3208da7b888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:32.972904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d3277-ed18-4b0c-ab1b-4c20c50389c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:32.973547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3277-ed18-4b0c-ab1b-4c20c50389c4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:32.973819 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3277-ed18-4b0c-ab1b-4c20c50389c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:32.974087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d3277-ed18-4b0c-ab1b-4c20c50389c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:32.984947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d3277-ed18-4b0c-ab1b-4c20c50389c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:32.986471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d3277-ed18-4b0c-ab1b-4c20c50389c4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-fa5d3277-ed18-4b0c-ab1b-4c20c50389c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:33.002355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:33.003436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:33.005213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:33.005213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:33.005213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:33.009246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 443/892] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:33.010770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d1fb5-edc5-4665-8786-92c364aac768 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.011056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d1fb5-edc5-4665-8786-92c364aac768 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.011331 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d1fb5-edc5-4665-8786-92c364aac768 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.015674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 444/893] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:33.015674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d1fb5-edc5-4665-8786-92c364aac768 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:33.043673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 450/894] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:33.043673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0ebbc-c703-4e82-9ff2-d906758697a8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.043673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0ebbc-c703-4e82-9ff2-d906758697a8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.043673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0ebbc-c703-4e82-9ff2-d906758697a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.066818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c4d0ebbc-c703-4e82-9ff2-d906758697a8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:33.068172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c4d0ebbc-c703-4e82-9ff2-d906758697a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:33.142487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d1fb5-edc5-4665-8786-92c364aac768 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:33.154360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 451/895] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:33 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:33.187797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb3401-8a5a-4a78-a7cd-8960d2e9837b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:33.188802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb3401-8a5a-4a78-a7cd-8960d2e9837b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.189329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb3401-8a5a-4a78-a7cd-8960d2e9837b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.189803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb3401-8a5a-4a78-a7cd-8960d2e9837b None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.190716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb3401-8a5a-4a78-a7cd-8960d2e9837b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:33.191325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb3401-8a5a-4a78-a7cd-8960d2e9837b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:33.193027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fb3401-8a5a-4a78-a7cd-8960d2e9837b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:33.203781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:33.207668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 445/896] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:33.213705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0ebbc-c703-4e82-9ff2-d906758697a8 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:33.280066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:33.331254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0ebbc-c703-4e82-9ff2-d906758697a8 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:33.373450 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e9905-02a8-421b-a976-f76a6219f091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:33.374155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e9905-02a8-421b-a976-f76a6219f091 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.374436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e9905-02a8-421b-a976-f76a6219f091 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.374709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e9905-02a8-421b-a976-f76a6219f091 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.376215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb3401-8a5a-4a78-a7cd-8960d2e9837b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:33.387261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e9905-02a8-421b-a976-f76a6219f091 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:33.389348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e9905-02a8-421b-a976-f76a6219f091 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-578e9905-02a8-421b-a976-f76a6219f091 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:33.404406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:33.405345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:33.406209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:33.406209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:33.406209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:33.406209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:33.406209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:33.406209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:33.406209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:33.406209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:33.406629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 452/897] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:33.414982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e9905-02a8-421b-a976-f76a6219f091 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:33.522279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb3401-8a5a-4a78-a7cd-8960d2e9837b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:33.555675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e9905-02a8-421b-a976-f76a6219f091 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:33.557577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189089c-678d-4f0f-a5e7-15edb4759839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:33.558205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189089c-678d-4f0f-a5e7-15edb4759839 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.558479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189089c-678d-4f0f-a5e7-15edb4759839 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.558743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189089c-678d-4f0f-a5e7-15edb4759839 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.559320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189089c-678d-4f0f-a5e7-15edb4759839 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:33.560730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189089c-678d-4f0f-a5e7-15edb4759839 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-1189089c-678d-4f0f-a5e7-15edb4759839 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:33.570082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:33.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:33.571923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 446/898] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:01:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:33.572705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189089c-678d-4f0f-a5e7-15edb4759839 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:33.579074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:33.601630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa5c1b-268b-4cf2-a0fc-3139afa69baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:33.602244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa5c1b-268b-4cf2-a0fc-3139afa69baf None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.602691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa5c1b-268b-4cf2-a0fc-3139afa69baf None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.602968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa5c1b-268b-4cf2-a0fc-3139afa69baf None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.604036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa5c1b-268b-4cf2-a0fc-3139afa69baf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:33.604343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa5c1b-268b-4cf2-a0fc-3139afa69baf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:33.605792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa5c1b-268b-4cf2-a0fc-3139afa69baf None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:33.623466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 453/899] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:33.623466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa5c1b-268b-4cf2-a0fc-3139afa69baf None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:33.644753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:33.691782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189089c-678d-4f0f-a5e7-15edb4759839 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:33.727059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa3710-a4d8-4d89-9e64-8f99b71f48a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:33.727847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa3710-a4d8-4d89-9e64-8f99b71f48a1 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.727960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa3710-a4d8-4d89-9e64-8f99b71f48a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.728259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fa3710-a4d8-4d89-9e64-8f99b71f48a1 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.729169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa3710-a4d8-4d89-9e64-8f99b71f48a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:33.730539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fa3710-a4d8-4d89-9e64-8f99b71f48a1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:33.730827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa5c1b-268b-4cf2-a0fc-3139afa69baf None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:33.746645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 447/900] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:01:33 2022] POST /identity/v3/services => generated 241 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:33.747864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa3710-a4d8-4d89-9e64-8f99b71f48a1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:33.788733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40138ca-c8ae-4f59-8067-e66db077ae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:33.789801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40138ca-c8ae-4f59-8067-e66db077ae84 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.790308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40138ca-c8ae-4f59-8067-e66db077ae84 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.790923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40138ca-c8ae-4f59-8067-e66db077ae84 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.792567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40138ca-c8ae-4f59-8067-e66db077ae84 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:33.793148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40138ca-c8ae-4f59-8067-e66db077ae84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:33.795113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40138ca-c8ae-4f59-8067-e66db077ae84 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:33.803125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 454/901] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:01:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:33.809486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40138ca-c8ae-4f59-8067-e66db077ae84 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:33.872146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:33.893207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fa3710-a4d8-4d89-9e64-8f99b71f48a1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:33.914811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:33.936429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0fcc51-cf10-492e-841c-ad5a40618d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:33.937142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fcc51-cf10-492e-841c-ad5a40618d33 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.937491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fcc51-cf10-492e-841c-ad5a40618d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.937771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0fcc51-cf10-492e-841c-ad5a40618d33 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.938511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fcc51-cf10-492e-841c-ad5a40618d33 None admin] RBAC: Adding query filter params (role.id=823d0de8de634181af75dabe48e834bb, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=d75640e511e64661826c8b8d7aac2ce7) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:33.938822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fcc51-cf10-492e-841c-ad5a40618d33 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:33.940313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0fcc51-cf10-492e-841c-ad5a40618d33 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:33.946544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 448/902] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:33 2022] GET /identity/v3/role_assignments?user.id=d75640e511e64661826c8b8d7aac2ce7&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=823d0de8de634181af75dabe48e834bb => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:33.947734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2df9a-63ff-4ace-930a-00d23a870f24 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.948015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2df9a-63ff-4ace-930a-00d23a870f24 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.948286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c2df9a-63ff-4ace-930a-00d23a870f24 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.949588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 449/903] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:33.955811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb9b9-b5c4-4b76-8170-9ba56429c11b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:33.956098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb9b9-b5c4-4b76-8170-9ba56429c11b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:33.957137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dfb9b9-b5c4-4b76-8170-9ba56429c11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:33.962367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40138ca-c8ae-4f59-8067-e66db077ae84 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:33.980090 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-23dfb9b9-b5c4-4b76-8170-9ba56429c11b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:33.981440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-23dfb9b9-b5c4-4b76-8170-9ba56429c11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:34.019300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab524b09-7d4d-4f82-b299-913b447f07a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:34.019967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab524b09-7d4d-4f82-b299-913b447f07a8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:34.020244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab524b09-7d4d-4f82-b299-913b447f07a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:34.020510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab524b09-7d4d-4f82-b299-913b447f07a8 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:34.021153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab524b09-7d4d-4f82-b299-913b447f07a8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:34.021521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab524b09-7d4d-4f82-b299-913b447f07a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:34.022986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab524b09-7d4d-4f82-b299-913b447f07a8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:34.029152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 455/904] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:01:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:34.036616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab524b09-7d4d-4f82-b299-913b447f07a8 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:34.078877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 450/905] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:34.151085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:34.154449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab524b09-7d4d-4f82-b299-913b447f07a8 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:34.193177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c62ea-a262-4cd0-8288-f4091ac9b8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:34.194106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c62ea-a262-4cd0-8288-f4091ac9b8c5 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:34.194378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c62ea-a262-4cd0-8288-f4091ac9b8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:34.194646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181c62ea-a262-4cd0-8288-f4091ac9b8c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:34.195343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c62ea-a262-4cd0-8288-f4091ac9b8c5 None admin] RBAC: Adding query filter params (interface=public, service_id=4c3202b79c4b459699aa475707d32fd6, region_id=RegionOne) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:34.195687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c62ea-a262-4cd0-8288-f4091ac9b8c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:34.197103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181c62ea-a262-4cd0-8288-f4091ac9b8c5 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:34.207610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 456/906] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 10 14:01:34 2022] GET /identity/v3/endpoints?service_id=4c3202b79c4b459699aa475707d32fd6&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:34.208961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfb9b9-b5c4-4b76-8170-9ba56429c11b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:34.310575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dfb9b9-b5c4-4b76-8170-9ba56429c11b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:34.325295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:34.347333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c35e9-bc6f-47f6-8521-47dac31635de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:34.348079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c35e9-bc6f-47f6-8521-47dac31635de None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:34.348355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c35e9-bc6f-47f6-8521-47dac31635de None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:34.348714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c35e9-bc6f-47f6-8521-47dac31635de None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:34.349658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c35e9-bc6f-47f6-8521-47dac31635de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:34.349990 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c62ea-a262-4cd0-8288-f4091ac9b8c5 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:34.351072 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c35e9-bc6f-47f6-8521-47dac31635de None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:34.357797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 451/907] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 10 14:01:34 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:01:34.485689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181c62ea-a262-4cd0-8288-f4091ac9b8c5 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:34.523565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0cb1c-e80a-441a-a991-68051c2c1ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:34.524213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0cb1c-e80a-441a-a991-68051c2c1ff1 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:34.524488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0cb1c-e80a-441a-a991-68051c2c1ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:34.524753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0cb1c-e80a-441a-a991-68051c2c1ff1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:34.525349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0cb1c-e80a-441a-a991-68051c2c1ff1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:34.526851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0cb1c-e80a-441a-a991-68051c2c1ff1 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-96c0cb1c-e80a-441a-a991-68051c2c1ff1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:34.536998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:34.538043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:34.539736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 457/908] 10.222.0.26 () {56 vars in 1132 bytes} [Sat Sep 10 14:01:34 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:34.544453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c35e9-bc6f-47f6-8521-47dac31635de None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:34.654230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c35e9-bc6f-47f6-8521-47dac31635de None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:34.693509 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8656-ca32-4e55-bbf4-28aa5a9e9f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:34.694505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8656-ca32-4e55-bbf4-28aa5a9e9f7b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:34.694999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8656-ca32-4e55-bbf4-28aa5a9e9f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:34.695520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f8656-ca32-4e55-bbf4-28aa5a9e9f7b None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:34.696364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f8656-ca32-4e55-bbf4-28aa5a9e9f7b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:34.696876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f8656-ca32-4e55-bbf4-28aa5a9e9f7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:34.698445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f8656-ca32-4e55-bbf4-28aa5a9e9f7b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:34.704268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 452/909] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:34 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:34.714689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0cb1c-e80a-441a-a991-68051c2c1ff1 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:34.741507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:34.822479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0cb1c-e80a-441a-a991-68051c2c1ff1 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:34.858577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6694e5-28c3-41ba-9158-b5b15900dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:34.859179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6694e5-28c3-41ba-9158-b5b15900dd66 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:34.859467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6694e5-28c3-41ba-9158-b5b15900dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:34.859748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6694e5-28c3-41ba-9158-b5b15900dd66 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:34.860372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6694e5-28c3-41ba-9158-b5b15900dd66 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:34.860671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6694e5-28c3-41ba-9158-b5b15900dd66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:34.862045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6694e5-28c3-41ba-9158-b5b15900dd66 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:34.867834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 458/910] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:34 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:34.874057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8656-ca32-4e55-bbf4-28aa5a9e9f7b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:34.920658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:34.982062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f8656-ca32-4e55-bbf4-28aa5a9e9f7b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:35.017284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e09e1f-db3b-4e14-8f94-a93bdb26184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:35.017815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e09e1f-db3b-4e14-8f94-a93bdb26184c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:35.018096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e09e1f-db3b-4e14-8f94-a93bdb26184c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:35.018372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e09e1f-db3b-4e14-8f94-a93bdb26184c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:35.019865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e09e1f-db3b-4e14-8f94-a93bdb26184c None admin] RBAC: Adding query filter params (interface=public, service_id=973b31cd3258491fbe1a3e2718811f74, region_id=RegionOne) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:35.020441 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e09e1f-db3b-4e14-8f94-a93bdb26184c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:35.021823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e09e1f-db3b-4e14-8f94-a93bdb26184c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:35.027318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 453/911] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 10 14:01:34 2022] GET /identity/v3/endpoints?service_id=973b31cd3258491fbe1a3e2718811f74&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:35.033014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6694e5-28c3-41ba-9158-b5b15900dd66 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:35.128724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:35.138049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6694e5-28c3-41ba-9158-b5b15900dd66 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:35.179001 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c168c7f-764f-4a54-b4e3-86859051b372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:35.179651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c168c7f-764f-4a54-b4e3-86859051b372 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:35.179931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c168c7f-764f-4a54-b4e3-86859051b372 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:35.180201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c168c7f-764f-4a54-b4e3-86859051b372 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:35.180800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c168c7f-764f-4a54-b4e3-86859051b372 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:35.182170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c168c7f-764f-4a54-b4e3-86859051b372 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:35.188714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 459/912] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 10 14:01:35 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:01:36.126447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43981102-d401-4574-85d9-719cbbe85130 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:36.128743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43981102-d401-4574-85d9-719cbbe85130 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:36.129414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43981102-d401-4574-85d9-719cbbe85130 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:36.131242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 454/913] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:36.136781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bc473-b466-4934-91af-1fd9d9c107a7 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:36.137576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bc473-b466-4934-91af-1fd9d9c107a7 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:36.138046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bc473-b466-4934-91af-1fd9d9c107a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:36.157922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d72bc473-b466-4934-91af-1fd9d9c107a7 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:36.159424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d72bc473-b466-4934-91af-1fd9d9c107a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:36.201982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2473b-04e5-442c-b1b6-8310a90f7c33 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:36.202296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2473b-04e5-442c-b1b6-8310a90f7c33 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:36.202528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf2473b-04e5-442c-b1b6-8310a90f7c33 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:36.204148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 455/914] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:36.209679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba410df-b23e-4b2b-85b4-51c9aeb2159f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:36.209963 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba410df-b23e-4b2b-85b4-51c9aeb2159f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:36.210235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba410df-b23e-4b2b-85b4-51c9aeb2159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:36.228663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-cba410df-b23e-4b2b-85b4-51c9aeb2159f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:36.229870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cba410df-b23e-4b2b-85b4-51c9aeb2159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:36.246696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 460/915] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:36.302633 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 456/916] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:36.532398 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bc473-b466-4934-91af-1fd9d9c107a7 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:36.589094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba410df-b23e-4b2b-85b4-51c9aeb2159f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:36.627813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:36.630817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:36.649179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bc473-b466-4934-91af-1fd9d9c107a7 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:36.691801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17596960-4fd9-432b-a250-b0927e25bdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:36.692988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17596960-4fd9-432b-a250-b0927e25bdc7 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:36.693263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17596960-4fd9-432b-a250-b0927e25bdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:36.693534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17596960-4fd9-432b-a250-b0927e25bdc7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:36.694111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17596960-4fd9-432b-a250-b0927e25bdc7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:36.696593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17596960-4fd9-432b-a250-b0927e25bdc7 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-17596960-4fd9-432b-a250-b0927e25bdc7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:36.704403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:36.705411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:36.706220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 461/917] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:36.707999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be51a9-a19b-4f4f-97be-07d3b72119d4 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:36.707999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be51a9-a19b-4f4f-97be-07d3b72119d4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:36.708131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be51a9-a19b-4f4f-97be-07d3b72119d4 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:36.713742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 462/918] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:36.714531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be51a9-a19b-4f4f-97be-07d3b72119d4 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:36.721888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba410df-b23e-4b2b-85b4-51c9aeb2159f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:36.761783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2674135-a1ac-45b3-b23c-194c3d1fcaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:36.762517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2674135-a1ac-45b3-b23c-194c3d1fcaa0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:36.762793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2674135-a1ac-45b3-b23c-194c3d1fcaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:36.763062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2674135-a1ac-45b3-b23c-194c3d1fcaa0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:36.774434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2674135-a1ac-45b3-b23c-194c3d1fcaa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-b2674135-a1ac-45b3-b23c-194c3d1fcaa0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:36.779729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:36.780686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:36.781516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 457/919] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 10 14:01:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:36.782716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df9a8d-430b-4ded-8906-ce62ff8675b4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:36.782716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df9a8d-430b-4ded-8906-ce62ff8675b4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:36.782716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64df9a8d-430b-4ded-8906-ce62ff8675b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:36.802125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-64df9a8d-430b-4ded-8906-ce62ff8675b4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:36.803316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-64df9a8d-430b-4ded-8906-ce62ff8675b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:36.827308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be51a9-a19b-4f4f-97be-07d3b72119d4 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:36.865991 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed768b6-a6bf-441a-8784-33fc4dd4c6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:36.866599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed768b6-a6bf-441a-8784-33fc4dd4c6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:36.866876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed768b6-a6bf-441a-8784-33fc4dd4c6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:36.867414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed768b6-a6bf-441a-8784-33fc4dd4c6a8 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:36.867791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed768b6-a6bf-441a-8784-33fc4dd4c6a8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:36.868090 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed768b6-a6bf-441a-8784-33fc4dd4c6a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:36.869456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed768b6-a6bf-441a-8784-33fc4dd4c6a8 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:36.879307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 463/920] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 10 14:01:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:36.879492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 458/921] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:36.879611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed768b6-a6bf-441a-8784-33fc4dd4c6a8 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:36.881180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df9a8d-430b-4ded-8906-ce62ff8675b4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:36.899163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:36.919843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:37.000453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed768b6-a6bf-441a-8784-33fc4dd4c6a8 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.022781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64df9a8d-430b-4ded-8906-ce62ff8675b4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.041645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905ab83-7cfc-4482-8bce-0e01c5cbc604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.042810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905ab83-7cfc-4482-8bce-0e01c5cbc604 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.043182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905ab83-7cfc-4482-8bce-0e01c5cbc604 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.043835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905ab83-7cfc-4482-8bce-0e01c5cbc604 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.044599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905ab83-7cfc-4482-8bce-0e01c5cbc604 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:37.045001 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905ab83-7cfc-4482-8bce-0e01c5cbc604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.046476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6905ab83-7cfc-4482-8bce-0e01c5cbc604 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.060189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 464/922] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 10 14:01:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:37.062266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905ab83-7cfc-4482-8bce-0e01c5cbc604 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:37.070478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc57db-a91b-4e14-a3d9-38d32f64e479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.071187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc57db-a91b-4e14-a3d9-38d32f64e479 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.071556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc57db-a91b-4e14-a3d9-38d32f64e479 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.071940 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc57db-a91b-4e14-a3d9-38d32f64e479 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.072641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc57db-a91b-4e14-a3d9-38d32f64e479 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:37.073004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc57db-a91b-4e14-a3d9-38d32f64e479 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.074539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cc57db-a91b-4e14-a3d9-38d32f64e479 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.084907 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 459/923] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 10 14:01:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:37.086594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc57db-a91b-4e14-a3d9-38d32f64e479 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:37.153776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:37.181713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:37.182225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905ab83-7cfc-4482-8bce-0e01c5cbc604 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.195659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc57db-a91b-4e14-a3d9-38d32f64e479 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.216806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb453e-64e3-44df-a0b0-b219dc09b5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.217438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb453e-64e3-44df-a0b0-b219dc09b5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.217718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb453e-64e3-44df-a0b0-b219dc09b5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.217987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fb453e-64e3-44df-a0b0-b219dc09b5e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.229848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb453e-64e3-44df-a0b0-b219dc09b5e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.230941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3546f-2ec7-4e7f-a5e1-d20f3bd356f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.231311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fb453e-64e3-44df-a0b0-b219dc09b5e1 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.231573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3546f-2ec7-4e7f-a5e1-d20f3bd356f0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.231846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3546f-2ec7-4e7f-a5e1-d20f3bd356f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.232109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3546f-2ec7-4e7f-a5e1-d20f3bd356f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.232797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3546f-2ec7-4e7f-a5e1-d20f3bd356f0 None admin] RBAC: Adding query filter params (interface=public, service_id=11febbccc7fc466ea0c7c846ad3e5066, region_id=RegionOne) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:37.233158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3546f-2ec7-4e7f-a5e1-d20f3bd356f0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.234442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3546f-2ec7-4e7f-a5e1-d20f3bd356f0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.240634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 460/924] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 10 14:01:37 2022] GET /identity/v3/endpoints?service_id=11febbccc7fc466ea0c7c846ad3e5066&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:37.244384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3546f-2ec7-4e7f-a5e1-d20f3bd356f0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-49fb453e-64e3-44df-a0b0-b219dc09b5e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:37.247399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:37.248369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:37.249242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:37.249242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:37.249242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:37.253343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 465/925] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:37.254095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb453e-64e3-44df-a0b0-b219dc09b5e1 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:37.358517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3546f-2ec7-4e7f-a5e1-d20f3bd356f0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.366794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fb453e-64e3-44df-a0b0-b219dc09b5e1 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.395543 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6ce40-c842-4695-b3af-591b3a26d705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.396182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6ce40-c842-4695-b3af-591b3a26d705 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.396457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6ce40-c842-4695-b3af-591b3a26d705 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.396589 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:37.396791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b6ce40-c842-4695-b3af-591b3a26d705 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.397313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6ce40-c842-4695-b3af-591b3a26d705 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.398710 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b6ce40-c842-4695-b3af-591b3a26d705 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.405750 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fae095-e5ec-40cf-b67a-70407e66867e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.406355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fae095-e5ec-40cf-b67a-70407e66867e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.406633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fae095-e5ec-40cf-b67a-70407e66867e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.406907 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fae095-e5ec-40cf-b67a-70407e66867e None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.407557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fae095-e5ec-40cf-b67a-70407e66867e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a3b6ce40-c842-4695-b3af-591b3a26d705 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:37.407965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:37.408901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 10 14:01:37.409766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:37.411102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fae095-e5ec-40cf-b67a-70407e66867e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.414036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 461/926] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:01:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:37.414894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6ce40-c842-4695-b3af-591b3a26d705 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:37.422294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 466/927] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 10 14:01:37 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:01:37.422294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fae095-e5ec-40cf-b67a-70407e66867e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:37.442322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:37.538275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b6ce40-c842-4695-b3af-591b3a26d705 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.542448 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fae095-e5ec-40cf-b67a-70407e66867e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.577559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7c194-eddc-4ed9-8016-3ba6a7fdb3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.578173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7c194-eddc-4ed9-8016-3ba6a7fdb3de None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.578444 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7c194-eddc-4ed9-8016-3ba6a7fdb3de None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.578706 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7c194-eddc-4ed9-8016-3ba6a7fdb3de None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.579351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7c194-eddc-4ed9-8016-3ba6a7fdb3de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:37.579660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6783b-2253-42bd-98e8-41ac3005f9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.579773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7c194-eddc-4ed9-8016-3ba6a7fdb3de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.580267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6783b-2253-42bd-98e8-41ac3005f9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.580536 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6783b-2253-42bd-98e8-41ac3005f9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.580799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6783b-2253-42bd-98e8-41ac3005f9f4 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.581040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f7c194-eddc-4ed9-8016-3ba6a7fdb3de None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.582002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6783b-2253-42bd-98e8-41ac3005f9f4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:37.582297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6783b-2253-42bd-98e8-41ac3005f9f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.583792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6783b-2253-42bd-98e8-41ac3005f9f4 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.589156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 467/928] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:01:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:37.590424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dff790-bffb-43d9-bcc8-2c99ef4d34b6 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.590703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dff790-bffb-43d9-bcc8-2c99ef4d34b6 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.590968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dff790-bffb-43d9-bcc8-2c99ef4d34b6 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.592178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 468/929] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:37.597504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dff790-bffb-43d9-bcc8-2c99ef4d34b6 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:37.599970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 462/930] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:37.601219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642c5e1-5309-4732-bea6-2f36c4237ca2 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.601501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642c5e1-5309-4732-bea6-2f36c4237ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.601770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9642c5e1-5309-4732-bea6-2f36c4237ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.623311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9642c5e1-5309-4732-bea6-2f36c4237ca2 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:37.624546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9642c5e1-5309-4732-bea6-2f36c4237ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:37.662824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:37.700353 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 463/931] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:37.701305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642c5e1-5309-4732-bea6-2f36c4237ca2 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:37.713711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:37.724372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dff790-bffb-43d9-bcc8-2c99ef4d34b6 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.766581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe655cb5-382e-438b-9b28-49123442732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.767630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe655cb5-382e-438b-9b28-49123442732e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.767914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe655cb5-382e-438b-9b28-49123442732e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.768185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe655cb5-382e-438b-9b28-49123442732e None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.769669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe655cb5-382e-438b-9b28-49123442732e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:37.770006 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe655cb5-382e-438b-9b28-49123442732e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.771851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe655cb5-382e-438b-9b28-49123442732e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.779074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 469/932] 10.222.0.26 () {56 vars in 1114 bytes} [Sat Sep 10 14:01:37 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:37.787482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe655cb5-382e-438b-9b28-49123442732e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:37.821162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9642c5e1-5309-4732-bea6-2f36c4237ca2 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.861065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009b047-cf11-4a52-99dd-f8476c932c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.861861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009b047-cf11-4a52-99dd-f8476c932c53 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.862235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009b047-cf11-4a52-99dd-f8476c932c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.862914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c009b047-cf11-4a52-99dd-f8476c932c53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.872349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009b047-cf11-4a52-99dd-f8476c932c53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.874443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c009b047-cf11-4a52-99dd-f8476c932c53 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c009b047-cf11-4a52-99dd-f8476c932c53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:37.891700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:37.892680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:37.893606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:37.893606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:37.893606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:37.893606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:37.893606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:37.893606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:37.893606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:37.893606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:37.893606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:37.893606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:37.895608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 464/933] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 10 14:01:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:37.899799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009b047-cf11-4a52-99dd-f8476c932c53 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:37.905936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe655cb5-382e-438b-9b28-49123442732e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:37.927765 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:37.943808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db41268-7785-4d47-8740-80105b130f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:37.944451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db41268-7785-4d47-8740-80105b130f8c None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:37.944729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db41268-7785-4d47-8740-80105b130f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:37.945000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db41268-7785-4d47-8740-80105b130f8c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:37.945671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db41268-7785-4d47-8740-80105b130f8c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:37.947550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db41268-7785-4d47-8740-80105b130f8c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:37.977289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 470/934] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:01:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:01:37.996340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:38.025387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db41268-7785-4d47-8740-80105b130f8c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:38.045567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c009b047-cf11-4a52-99dd-f8476c932c53 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:38.100273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d62e8d-6d99-4b59-a5ea-cef8bb0a3c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:38.101333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d62e8d-6d99-4b59-a5ea-cef8bb0a3c4b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:38.101821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d62e8d-6d99-4b59-a5ea-cef8bb0a3c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:38.102298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d62e8d-6d99-4b59-a5ea-cef8bb0a3c4b None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:38.103266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d62e8d-6d99-4b59-a5ea-cef8bb0a3c4b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:38.103832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d62e8d-6d99-4b59-a5ea-cef8bb0a3c4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:38.105586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d62e8d-6d99-4b59-a5ea-cef8bb0a3c4b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:38.131890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 465/935] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 10 14:01:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:38.134248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d62e8d-6d99-4b59-a5ea-cef8bb0a3c4b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:38.158768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db41268-7785-4d47-8740-80105b130f8c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:38.199673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc7bf8-bfaf-4513-80b0-820c6ab698cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:38.200311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7bf8-bfaf-4513-80b0-820c6ab698cb None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:38.200582 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7bf8-bfaf-4513-80b0-820c6ab698cb None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:38.200850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cc7bf8-bfaf-4513-80b0-820c6ab698cb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:38.201437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc7bf8-bfaf-4513-80b0-820c6ab698cb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:38.202799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cc7bf8-bfaf-4513-80b0-820c6ab698cb None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-94cc7bf8-bfaf-4513-80b0-820c6ab698cb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:38.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:38.216034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:38.216955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 471/936] 10.222.0.26 () {56 vars in 1132 bytes} [Sat Sep 10 14:01:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:38.223595 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc7bf8-bfaf-4513-80b0-820c6ab698cb None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:38.256446 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d62e8d-6d99-4b59-a5ea-cef8bb0a3c4b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:38.283888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:38.286095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:38.302779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ea91e-e76b-4f79-bd6c-f03e44eef4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:38.303783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ea91e-e76b-4f79-bd6c-f03e44eef4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:38.304310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ea91e-e76b-4f79-bd6c-f03e44eef4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:38.304788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ea91e-e76b-4f79-bd6c-f03e44eef4b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:38.315565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ea91e-e76b-4f79-bd6c-f03e44eef4b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:38.317631 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29ea91e-e76b-4f79-bd6c-f03e44eef4b5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d29ea91e-e76b-4f79-bd6c-f03e44eef4b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:38.334975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:38.335999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:38.336896 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:38.336896 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:38.336896 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:38.339084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 466/937] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:38.344890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ea91e-e76b-4f79-bd6c-f03e44eef4b5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:38.351732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cc7bf8-bfaf-4513-80b0-820c6ab698cb None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:38.403427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bb7b7-e405-482b-97f5-9fd7e6c4290b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:38.404068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bb7b7-e405-482b-97f5-9fd7e6c4290b None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:38.404348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bb7b7-e405-482b-97f5-9fd7e6c4290b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:38.404616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9bb7b7-e405-482b-97f5-9fd7e6c4290b None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:38.405251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9bb7b7-e405-482b-97f5-9fd7e6c4290b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:38.405552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9bb7b7-e405-482b-97f5-9fd7e6c4290b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:38.406916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9bb7b7-e405-482b-97f5-9fd7e6c4290b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:38.417057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 472/938] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:38.418930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bb7b7-e405-482b-97f5-9fd7e6c4290b None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:38.483092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29ea91e-e76b-4f79-bd6c-f03e44eef4b5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:38.520434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e23f58-632e-4a07-92f1-9f5762a3ff2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:38.521178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e23f58-632e-4a07-92f1-9f5762a3ff2d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:38.521559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e23f58-632e-4a07-92f1-9f5762a3ff2d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:38.521952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e23f58-632e-4a07-92f1-9f5762a3ff2d None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:38.522703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e23f58-632e-4a07-92f1-9f5762a3ff2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:38.523106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e23f58-632e-4a07-92f1-9f5762a3ff2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:38.524650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e23f58-632e-4a07-92f1-9f5762a3ff2d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:38.537649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 467/939] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:38.543152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e23f58-632e-4a07-92f1-9f5762a3ff2d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:38.553848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9bb7b7-e405-482b-97f5-9fd7e6c4290b None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:38.557563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:38.578722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:38.604811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907d987-d5bc-4b7a-a478-b9d697d47dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:38.606191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907d987-d5bc-4b7a-a478-b9d697d47dfd None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:38.606901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907d987-d5bc-4b7a-a478-b9d697d47dfd None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:38.607377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907d987-d5bc-4b7a-a478-b9d697d47dfd None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:38.608462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907d987-d5bc-4b7a-a478-b9d697d47dfd None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:38.608953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907d987-d5bc-4b7a-a478-b9d697d47dfd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:38.611153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907d987-d5bc-4b7a-a478-b9d697d47dfd None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:38.624076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 473/940] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:38 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:38.625786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907d987-d5bc-4b7a-a478-b9d697d47dfd None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:38.664578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e23f58-632e-4a07-92f1-9f5762a3ff2d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:38.700505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada48b7-840f-4d4d-ba43-0e52e40c77d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:38.701135 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada48b7-840f-4d4d-ba43-0e52e40c77d2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:38.701409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada48b7-840f-4d4d-ba43-0e52e40c77d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:38.701677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada48b7-840f-4d4d-ba43-0e52e40c77d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:38.708918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada48b7-840f-4d4d-ba43-0e52e40c77d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:38.710371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada48b7-840f-4d4d-ba43-0e52e40c77d2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-eada48b7-840f-4d4d-ba43-0e52e40c77d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:38.722789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:38.723750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:38.724654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:38.724654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:38.724654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:38.724654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:38.724654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:38.724654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:38.724654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:38.724654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:38.724654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 468/941] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:38.732454 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada48b7-840f-4d4d-ba43-0e52e40c77d2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:38.758294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907d987-d5bc-4b7a-a478-b9d697d47dfd None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:38.802826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd1a3d-eb83-4710-bb4a-4da5d25926c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:38.803879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd1a3d-eb83-4710-bb4a-4da5d25926c2 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:38.804894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd1a3d-eb83-4710-bb4a-4da5d25926c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:38.805357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd1a3d-eb83-4710-bb4a-4da5d25926c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:38.806495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd1a3d-eb83-4710-bb4a-4da5d25926c2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:38.808327 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd1a3d-eb83-4710-bb4a-4da5d25926c2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:38.838225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 474/942] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:01:38 2022] POST /identity/v3/endpoints => generated 347 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:38.852104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:38.864612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada48b7-840f-4d4d-ba43-0e52e40c77d2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:38.912049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09030f9b-54f4-4e55-b183-f816885a5b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:38.912704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09030f9b-54f4-4e55-b183-f816885a5b2f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:38.912989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09030f9b-54f4-4e55-b183-f816885a5b2f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:38.913264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09030f9b-54f4-4e55-b183-f816885a5b2f None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:38.913961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09030f9b-54f4-4e55-b183-f816885a5b2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:38.914268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09030f9b-54f4-4e55-b183-f816885a5b2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:38.915835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09030f9b-54f4-4e55-b183-f816885a5b2f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:38.935862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 469/943] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:38.937880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd1a3d-eb83-4710-bb4a-4da5d25926c2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:38.970887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:39.062548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd1a3d-eb83-4710-bb4a-4da5d25926c2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:39.102776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c190f1-05a6-4c66-9d52-b99d02e216ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:39.103455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c190f1-05a6-4c66-9d52-b99d02e216ab None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:39.103742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c190f1-05a6-4c66-9d52-b99d02e216ab None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:39.104014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c190f1-05a6-4c66-9d52-b99d02e216ab None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/d75640e511e64661826c8b8d7aac2ce7/roles/823d0de8de634181af75dabe48e834bb` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:39.106412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c190f1-05a6-4c66-9d52-b99d02e216ab None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=d75640e511e64661826c8b8d7aac2ce7, role_id=823d0de8de634181af75dabe48e834bb)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:39.107934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c190f1-05a6-4c66-9d52-b99d02e216ab None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:39.123598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 475/944] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:01:38 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/d75640e511e64661826c8b8d7aac2ce7/roles/823d0de8de634181af75dabe48e834bb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:01:39.668702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a241cd6-281d-429e-b9ac-1c39f5126d9b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:39.669252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a241cd6-281d-429e-b9ac-1c39f5126d9b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:39.669252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a241cd6-281d-429e-b9ac-1c39f5126d9b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:39.670816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 470/945] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:39.677008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78a5d7-bed2-4902-8520-82f0b7116523 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:39.677463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78a5d7-bed2-4902-8520-82f0b7116523 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:39.677549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78a5d7-bed2-4902-8520-82f0b7116523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:39.695935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-5a78a5d7-bed2-4902-8520-82f0b7116523 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:39.697056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5a78a5d7-bed2-4902-8520-82f0b7116523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:39.764980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 476/946] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:40.048377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a241cd6-281d-429e-b9ac-1c39f5126d9b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:40.160805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a241cd6-281d-429e-b9ac-1c39f5126d9b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:40.197761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5469d953-575c-484a-a07c-0ee5a4065a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:40.198684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469d953-575c-484a-a07c-0ee5a4065a5c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:40.198684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469d953-575c-484a-a07c-0ee5a4065a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:40.199238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5469d953-575c-484a-a07c-0ee5a4065a5c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:40.199560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469d953-575c-484a-a07c-0ee5a4065a5c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:40.200964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5469d953-575c-484a-a07c-0ee5a4065a5c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-5469d953-575c-484a-a07c-0ee5a4065a5c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:40.212475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:40.214956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:40.216962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 471/947] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:40.223470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78a5d7-bed2-4902-8520-82f0b7116523 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:40.236434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:40.326961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff12050-94fe-4a75-ae76-f1b6155457b1 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:40.327227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff12050-94fe-4a75-ae76-f1b6155457b1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:40.327415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff12050-94fe-4a75-ae76-f1b6155457b1 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:40.329000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 472/948] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:40.329637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78a5d7-bed2-4902-8520-82f0b7116523 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:40.334426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547c56-1dec-462e-bd42-c6313b78df4e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:40.334708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547c56-1dec-462e-bd42-c6313b78df4e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:40.334984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33547c56-1dec-462e-bd42-c6313b78df4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:40.355032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-33547c56-1dec-462e-bd42-c6313b78df4e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:40.356288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-33547c56-1dec-462e-bd42-c6313b78df4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:40.364633 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:40.375206 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8975a8d-2ce4-49c4-a71e-8c0fe8121207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:40.375851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8975a8d-2ce4-49c4-a71e-8c0fe8121207 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:40.376130 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8975a8d-2ce4-49c4-a71e-8c0fe8121207 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:40.376402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8975a8d-2ce4-49c4-a71e-8c0fe8121207 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:40.377215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8975a8d-2ce4-49c4-a71e-8c0fe8121207 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:40.377524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8975a8d-2ce4-49c4-a71e-8c0fe8121207 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:40.380084 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8975a8d-2ce4-49c4-a71e-8c0fe8121207 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:40.386173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 477/949] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 10 14:01:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:40.392771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8975a8d-2ce4-49c4-a71e-8c0fe8121207 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:40.451144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 473/950] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:40.484394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:40.507617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8975a8d-2ce4-49c4-a71e-8c0fe8121207 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:40.543286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0040d4-8778-4054-a0ec-e2c688729c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:40.544125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0040d4-8778-4054-a0ec-e2c688729c76 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:40.544567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0040d4-8778-4054-a0ec-e2c688729c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:40.545042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0040d4-8778-4054-a0ec-e2c688729c76 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:40.545846 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0040d4-8778-4054-a0ec-e2c688729c76 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:40.546287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0040d4-8778-4054-a0ec-e2c688729c76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:40.547807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0040d4-8778-4054-a0ec-e2c688729c76 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:40.554107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 478/951] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 10 14:01:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:40.560538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33547c56-1dec-462e-bd42-c6313b78df4e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:40.669187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33547c56-1dec-462e-bd42-c6313b78df4e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:40.709833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e40d38-e4b1-44bf-ba93-610567ebd163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:40.710445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40d38-e4b1-44bf-ba93-610567ebd163 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:40.710722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40d38-e4b1-44bf-ba93-610567ebd163 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:40.710991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40d38-e4b1-44bf-ba93-610567ebd163 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:40.711680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e40d38-e4b1-44bf-ba93-610567ebd163 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:40.713116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e40d38-e4b1-44bf-ba93-610567ebd163 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:40.730593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:40.738404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 474/952] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:01:40 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:40.756950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0040d4-8778-4054-a0ec-e2c688729c76 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:40.876588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0040d4-8778-4054-a0ec-e2c688729c76 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:40.922603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:40.923577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:40.924056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:40.924541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:40.925772 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:40.927473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:40.930318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: WARNING keystone.server.flask.common [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 10 14:01:40.940223 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:40.952909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.sql.core [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 14:01:40.952909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 14:01:40.952909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [parameters: {'id': 'c41ea052c5904e65b8acc32627918faf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 10 14:01:40.952909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81588) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c41ea052c5904e65b8acc32627918faf.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c41ea052c5904e65b8acc32627918faf. Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application conn.query(q) Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 14:01:40.959963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result.read() Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 10 14:01:40.960773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application next(self.gen) Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application yield resource Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application next(self.gen) Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise self.value Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application session.commit() Sep 10 14:01:40.961743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.session.flush() Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application compat.raise_( Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 14:01:40.962620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 14:01:40.964302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application conn.query(q) Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result.read() Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 14:01:40.966561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [parameters: {'id': 'c41ea052c5904e65b8acc32627918faf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:40.968874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:40.971275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:40.975274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:40.975274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 10 14:01:40.975274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 10 14:01:40.975274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:40.975274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:40.975274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 10 14:01:40.975274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 10 14:01:40.975274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c41ea052c5904e65b8acc32627918faf. Sep 10 14:01:40.975274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:40.975274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 479/953] 10.222.0.26 () {60 vars in 1140 bytes} [Sat Sep 10 14:01:40 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:40.976750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e40d38-e4b1-44bf-ba93-610567ebd163 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:41.077753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e40d38-e4b1-44bf-ba93-610567ebd163 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:41.117716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680a74c-375d-4d4e-85f1-32b33f21d161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:41.118363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680a74c-375d-4d4e-85f1-32b33f21d161 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:41.118636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680a74c-375d-4d4e-85f1-32b33f21d161 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:41.118902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680a74c-375d-4d4e-85f1-32b33f21d161 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:41.126392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680a74c-375d-4d4e-85f1-32b33f21d161 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:41.127964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6680a74c-375d-4d4e-85f1-32b33f21d161 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-6680a74c-375d-4d4e-85f1-32b33f21d161 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:41.140213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:41.141180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:41.142104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:41.142104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:41.142104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:41.142104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:41.142104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:41.142104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:41.142104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:41.142104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:41.142104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 475/954] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:41.148569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:41.215361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141176a5-8e9d-4e5b-a360-23f5caf09619 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:41.215685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141176a5-8e9d-4e5b-a360-23f5caf09619 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:41.215851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141176a5-8e9d-4e5b-a360-23f5caf09619 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:41.217445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 476/955] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:41.223423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212315f-d667-4804-bea9-8a8f396ff61b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:41.223733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212315f-d667-4804-bea9-8a8f396ff61b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:41.224010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212315f-d667-4804-bea9-8a8f396ff61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:41.242745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e212315f-d667-4804-bea9-8a8f396ff61b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:41.244025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e212315f-d667-4804-bea9-8a8f396ff61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:41.261144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceebf7d-7070-411c-a568-d32aee6000d4 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:41.306784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9458f3f0-b351-424c-b107-fe35d0b8fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:41.307582 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f3f0-b351-424c-b107-fe35d0b8fd0e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:41.307911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f3f0-b351-424c-b107-fe35d0b8fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:41.308207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9458f3f0-b351-424c-b107-fe35d0b8fd0e None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:41.309028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458f3f0-b351-424c-b107-fe35d0b8fd0e None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:41.309324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458f3f0-b351-424c-b107-fe35d0b8fd0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:41.310955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9458f3f0-b351-424c-b107-fe35d0b8fd0e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:41.316891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:41.343523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 477/957] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:41.343835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 480/957] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:41 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:41.386081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e14d6-e4b3-4445-addf-5e7c3540cbf1 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:41.386364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e14d6-e4b3-4445-addf-5e7c3540cbf1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:41.386641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597e14d6-e4b3-4445-addf-5e7c3540cbf1 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:41.388311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 478/958] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:41.394727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe870d-fe9b-4f4a-816a-364f699bc427 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:41.395292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe870d-fe9b-4f4a-816a-364f699bc427 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:41.395732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe870d-fe9b-4f4a-816a-364f699bc427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:41.416948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d6fe870d-fe9b-4f4a-816a-364f699bc427 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:41.418752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d6fe870d-fe9b-4f4a-816a-364f699bc427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:41.489646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:41.514196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 481/959] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:41.642463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e14d6-e4b3-4445-addf-5e7c3540cbf1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:41.750110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597e14d6-e4b3-4445-addf-5e7c3540cbf1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:41.788059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043033a1-3571-4cc0-9984-b214f758368b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:41.789072 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043033a1-3571-4cc0-9984-b214f758368b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:41.789599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043033a1-3571-4cc0-9984-b214f758368b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:41.790113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043033a1-3571-4cc0-9984-b214f758368b None admin] PATH_INFO: `/v3/services/compute` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:41.790994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043033a1-3571-4cc0-9984-b214f758368b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:41.792717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043033a1-3571-4cc0-9984-b214f758368b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-043033a1-3571-4cc0-9984-b214f758368b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:41.803078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:41.804343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 14:01:41.805195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:41.808168 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 479/960] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:41.814030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe870d-fe9b-4f4a-816a-364f699bc427 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:41.814784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043033a1-3571-4cc0-9984-b214f758368b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:41.873354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:41.924165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043033a1-3571-4cc0-9984-b214f758368b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:41.925628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe870d-fe9b-4f4a-816a-364f699bc427 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:41.961178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25fec0-7acc-4ba5-a6e2-0ec83ce91ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:41.961783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25fec0-7acc-4ba5-a6e2-0ec83ce91ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:41.962057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25fec0-7acc-4ba5-a6e2-0ec83ce91ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:41.962325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25fec0-7acc-4ba5-a6e2-0ec83ce91ce9 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:41.962950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25fec0-7acc-4ba5-a6e2-0ec83ce91ce9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:41.963244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25fec0-7acc-4ba5-a6e2-0ec83ce91ce9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:41.964658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c25fec0-7acc-4ba5-a6e2-0ec83ce91ce9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:41.965411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937543cd-976e-43c5-b536-a3ca3b79fdfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:41.966275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937543cd-976e-43c5-b536-a3ca3b79fdfe None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:41.966772 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937543cd-976e-43c5-b536-a3ca3b79fdfe None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:41.967246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937543cd-976e-43c5-b536-a3ca3b79fdfe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:41.974669 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 480/961] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:41.975879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25fec0-7acc-4ba5-a6e2-0ec83ce91ce9 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:41.979556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937543cd-976e-43c5-b536-a3ca3b79fdfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-937543cd-976e-43c5-b536-a3ca3b79fdfe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:41.984788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:41.985705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:01:41.986530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:41.990062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 482/962] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 10 14:01:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:41.996198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937543cd-976e-43c5-b536-a3ca3b79fdfe None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:42.057270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:42.126545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25fec0-7acc-4ba5-a6e2-0ec83ce91ce9 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:42.127407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937543cd-976e-43c5-b536-a3ca3b79fdfe None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:42.150378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:42.167221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d9a0-8fa4-4856-afcb-4fa12da17031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:42.167901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d9a0-8fa4-4856-afcb-4fa12da17031 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:42.168186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d9a0-8fa4-4856-afcb-4fa12da17031 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:42.168464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d9a0-8fa4-4856-afcb-4fa12da17031 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:42.169147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802d9a0-8fa4-4856-afcb-4fa12da17031 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:42.169477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802d9a0-8fa4-4856-afcb-4fa12da17031 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:42.170833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802d9a0-8fa4-4856-afcb-4fa12da17031 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:42.176003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599bfec-f0d3-4561-b009-454efc93ba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:42.177217 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599bfec-f0d3-4561-b009-454efc93ba28 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:42.177904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599bfec-f0d3-4561-b009-454efc93ba28 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:42.178320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599bfec-f0d3-4561-b009-454efc93ba28 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:42.179106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599bfec-f0d3-4561-b009-454efc93ba28 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:42.183983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 483/963] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 10 14:01:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:42.184114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599bfec-f0d3-4561-b009-454efc93ba28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:42.185285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599bfec-f0d3-4561-b009-454efc93ba28 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:42.185722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d9a0-8fa4-4856-afcb-4fa12da17031 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:42.195785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 481/964] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:41 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:42.303898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d9a0-8fa4-4856-afcb-4fa12da17031 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:42.313065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:42.344500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d627870-632c-4152-bf69-891548d4a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:42.345145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627870-632c-4152-bf69-891548d4a036 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:42.345685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627870-632c-4152-bf69-891548d4a036 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:42.345950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d627870-632c-4152-bf69-891548d4a036 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:42.357352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d627870-632c-4152-bf69-891548d4a036 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:42.358890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d627870-632c-4152-bf69-891548d4a036 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-8d627870-632c-4152-bf69-891548d4a036 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:42.376015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:42.376987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:42.377876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:42.377876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:42.377876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:42.378058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 484/965] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:42.384626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599bfec-f0d3-4561-b009-454efc93ba28 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:42.510284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599bfec-f0d3-4561-b009-454efc93ba28 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:42.547487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c6c2-fdf7-49a9-ac79-52cbfa7e40c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:42.548103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c6c2-fdf7-49a9-ac79-52cbfa7e40c8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:42.548379 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c6c2-fdf7-49a9-ac79-52cbfa7e40c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:42.548649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7325c6c2-fdf7-49a9-ac79-52cbfa7e40c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:42.549386 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325c6c2-fdf7-49a9-ac79-52cbfa7e40c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:42.549690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325c6c2-fdf7-49a9-ac79-52cbfa7e40c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:42.551715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7325c6c2-fdf7-49a9-ac79-52cbfa7e40c8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:42.564759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 482/966] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:42.570563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d627870-632c-4152-bf69-891548d4a036 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:42.673871 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d627870-632c-4152-bf69-891548d4a036 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:42.709590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4b9cc-87af-4872-8eb1-ac007b9346d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:42.710224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4b9cc-87af-4872-8eb1-ac007b9346d7 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:42.710495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4b9cc-87af-4872-8eb1-ac007b9346d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:42.710760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea4b9cc-87af-4872-8eb1-ac007b9346d7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:42.718406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4b9cc-87af-4872-8eb1-ac007b9346d7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:42.719935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea4b9cc-87af-4872-8eb1-ac007b9346d7 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-9ea4b9cc-87af-4872-8eb1-ac007b9346d7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:42.731707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:42.732663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:42.733603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:42.733603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:42.733603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:42.733603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:42.733603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:42.733603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:42.733603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:42.733603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:42.733603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 10 14:01:42.733603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:42.738081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 485/967] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 10 14:01:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:42.739507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c6c2-fdf7-49a9-ac79-52cbfa7e40c8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:42.783097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:42.847869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7325c6c2-fdf7-49a9-ac79-52cbfa7e40c8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:42.882994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f18e8f-5b6b-43eb-b255-8885aa74f3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:42.883627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f18e8f-5b6b-43eb-b255-8885aa74f3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:42.883906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f18e8f-5b6b-43eb-b255-8885aa74f3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:42.884180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f18e8f-5b6b-43eb-b255-8885aa74f3b9 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:42.885004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f18e8f-5b6b-43eb-b255-8885aa74f3b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:42.885315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f18e8f-5b6b-43eb-b255-8885aa74f3b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:42.886915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f18e8f-5b6b-43eb-b255-8885aa74f3b9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:42.908187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 483/968] 10.222.0.26 () {56 vars in 1149 bytes} [Sat Sep 10 14:01:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:42.911563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4b9cc-87af-4872-8eb1-ac007b9346d7 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:42.931757 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:43.020738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea4b9cc-87af-4872-8eb1-ac007b9346d7 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:43.056748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326fc9f-831a-4478-9e74-e9e16e43c010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:43.057360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326fc9f-831a-4478-9e74-e9e16e43c010 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:43.057638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326fc9f-831a-4478-9e74-e9e16e43c010 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:43.057909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326fc9f-831a-4478-9e74-e9e16e43c010 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:43.138309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326fc9f-831a-4478-9e74-e9e16e43c010 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:43.139740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8326fc9f-831a-4478-9e74-e9e16e43c010 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:43.163871 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:43.195288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c8c07-a448-4738-99e4-fed3e68be164 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:43.195997 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c8c07-a448-4738-99e4-fed3e68be164 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:43.196482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12c8c07-a448-4738-99e4-fed3e68be164 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:43.198277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 484/969] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:01:43.203429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66862e7-8963-4408-af68-2790cd4c89ba None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:43.203993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66862e7-8963-4408-af68-2790cd4c89ba None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:43.204476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66862e7-8963-4408-af68-2790cd4c89ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:43.224108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e66862e7-8963-4408-af68-2790cd4c89ba None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:43.225714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e66862e7-8963-4408-af68-2790cd4c89ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:43.245426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 486/970] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:42 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:01:43.252961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326fc9f-831a-4478-9e74-e9e16e43c010 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:43.303209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 485/971] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:43.358160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8326fc9f-831a-4478-9e74-e9e16e43c010 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:43.376375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:43.393993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a886f7-7c39-4b55-9b6b-dfed0a4260de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:43.394525 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a886f7-7c39-4b55-9b6b-dfed0a4260de None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:43.394803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a886f7-7c39-4b55-9b6b-dfed0a4260de None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:43.395075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a886f7-7c39-4b55-9b6b-dfed0a4260de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:43.405390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a886f7-7c39-4b55-9b6b-dfed0a4260de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:43.406840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a886f7-7c39-4b55-9b6b-dfed0a4260de None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-85a886f7-7c39-4b55-9b6b-dfed0a4260de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:43.421797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:43.422734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:43.423649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:43.423649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:43.423649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:43.423649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 487/972] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:43.431864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66862e7-8963-4408-af68-2790cd4c89ba None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:43.462266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:43.541492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66862e7-8963-4408-af68-2790cd4c89ba None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:43.580266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9e22c-c08f-49c4-a05c-619b92ac5457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:43.580883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9e22c-c08f-49c4-a05c-619b92ac5457 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:43.581162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9e22c-c08f-49c4-a05c-619b92ac5457 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:43.581961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b9e22c-c08f-49c4-a05c-619b92ac5457 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:43.582613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9e22c-c08f-49c4-a05c-619b92ac5457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:43.582917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9e22c-c08f-49c4-a05c-619b92ac5457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:43.584332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b9e22c-c08f-49c4-a05c-619b92ac5457 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:43.596736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 486/973] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:43.602198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9e22c-c08f-49c4-a05c-619b92ac5457 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:43.602516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a886f7-7c39-4b55-9b6b-dfed0a4260de None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:43.698814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:43.712768 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b9e22c-c08f-49c4-a05c-619b92ac5457 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:43.717297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a886f7-7c39-4b55-9b6b-dfed0a4260de None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:43.749102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d1f66-4059-4281-8f0e-6c50a705c761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:43.749728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1f66-4059-4281-8f0e-6c50a705c761 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:43.750007 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1f66-4059-4281-8f0e-6c50a705c761 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:43.750278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d1f66-4059-4281-8f0e-6c50a705c761 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:43.750861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d1f66-4059-4281-8f0e-6c50a705c761 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:43.752262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8d1f66-4059-4281-8f0e-6c50a705c761 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:43.754139 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2604e-b579-4521-9088-ba607f7550ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:43.754878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2604e-b579-4521-9088-ba607f7550ed None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:43.755157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2604e-b579-4521-9088-ba607f7550ed None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:43.755579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2604e-b579-4521-9088-ba607f7550ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-8b8d1f66-4059-4281-8f0e-6c50a705c761 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:43.761673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:43.762641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:43.763500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 487/974] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:43.764283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4dd63-071a-469b-8fc9-6f0117b228fa None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:43.764283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4dd63-071a-469b-8fc9-6f0117b228fa None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:43.765013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4dd63-071a-469b-8fc9-6f0117b228fa None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:43.765864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 488/975] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:43.768346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2604e-b579-4521-9088-ba607f7550ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:43.770628 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4dd63-071a-469b-8fc9-6f0117b228fa None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:43.770740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2604e-b579-4521-9088-ba607f7550ed None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-e7a2604e-b579-4521-9088-ba607f7550ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:43.782771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:43.783935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:43.785085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:43.785085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:43.785085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:43.785085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:43.785085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:43.785085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:43.785085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:43.785085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:43.788659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 488/976] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:43.789879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb9dec-6b00-41c5-a990-4ab24e94a309 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:43.790161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb9dec-6b00-41c5-a990-4ab24e94a309 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:43.790435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb9dec-6b00-41c5-a990-4ab24e94a309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:43.808539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-41eb9dec-6b00-41c5-a990-4ab24e94a309 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:43.809717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-41eb9dec-6b00-41c5-a990-4ab24e94a309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:43.834235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:43.880155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 489/977] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:43.881003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb9dec-6b00-41c5-a990-4ab24e94a309 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:43.900264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a4dd63-071a-469b-8fc9-6f0117b228fa None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:43.939957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e291b6-15b2-403a-a0a4-63c2fc67622d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:43.940731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e291b6-15b2-403a-a0a4-63c2fc67622d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:43.941901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e291b6-15b2-403a-a0a4-63c2fc67622d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:43.942127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e291b6-15b2-403a-a0a4-63c2fc67622d None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:43.942900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e291b6-15b2-403a-a0a4-63c2fc67622d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:43.943269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e291b6-15b2-403a-a0a4-63c2fc67622d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:43.944815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e291b6-15b2-403a-a0a4-63c2fc67622d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:43.954980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 489/978] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 10 14:01:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:43.956033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e291b6-15b2-403a-a0a4-63c2fc67622d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:43.977281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:44.013125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb9dec-6b00-41c5-a990-4ab24e94a309 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:44.052963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd315d0-dcbc-49d1-ba52-eab34293b358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:44.053858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd315d0-dcbc-49d1-ba52-eab34293b358 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:44.054285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd315d0-dcbc-49d1-ba52-eab34293b358 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:44.054705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd315d0-dcbc-49d1-ba52-eab34293b358 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:44.056072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd315d0-dcbc-49d1-ba52-eab34293b358 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:44.056527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd315d0-dcbc-49d1-ba52-eab34293b358 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:44.058496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd315d0-dcbc-49d1-ba52-eab34293b358 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:44.073098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 490/979] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:44.078878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd315d0-dcbc-49d1-ba52-eab34293b358 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:44.087485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e291b6-15b2-403a-a0a4-63c2fc67622d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:44.103900 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:44.124617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eec79-7e2d-44d4-a613-1c0fbd0a9c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:44.125685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eec79-7e2d-44d4-a613-1c0fbd0a9c13 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:44.126228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eec79-7e2d-44d4-a613-1c0fbd0a9c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:44.126746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124eec79-7e2d-44d4-a613-1c0fbd0a9c13 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:44.127615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eec79-7e2d-44d4-a613-1c0fbd0a9c13 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:44.128819 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eec79-7e2d-44d4-a613-1c0fbd0a9c13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:44.130452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124eec79-7e2d-44d4-a613-1c0fbd0a9c13 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:44.136548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 490/980] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 10 14:01:43 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:44.142543 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eec79-7e2d-44d4-a613-1c0fbd0a9c13 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:44.202549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd315d0-dcbc-49d1-ba52-eab34293b358 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:44.238016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247980c8-3e4b-42a5-8e7d-1c114b91047a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:44.238569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247980c8-3e4b-42a5-8e7d-1c114b91047a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:44.238845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247980c8-3e4b-42a5-8e7d-1c114b91047a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:44.239109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247980c8-3e4b-42a5-8e7d-1c114b91047a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:44.239877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247980c8-3e4b-42a5-8e7d-1c114b91047a None admin] RBAC: Adding query filter params (role.id=823d0de8de634181af75dabe48e834bb, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=d75640e511e64661826c8b8d7aac2ce7) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:44.240184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247980c8-3e4b-42a5-8e7d-1c114b91047a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:44.241682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247980c8-3e4b-42a5-8e7d-1c114b91047a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:44.243773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:44.248823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 491/981] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:44 2022] GET /identity/v3/role_assignments?user.id=d75640e511e64661826c8b8d7aac2ce7&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=823d0de8de634181af75dabe48e834bb => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:44.251603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247980c8-3e4b-42a5-8e7d-1c114b91047a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:44.278201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124eec79-7e2d-44d4-a613-1c0fbd0a9c13 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:44.329281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400538d0-210e-469d-a005-3aee7e31c4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:44.329898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400538d0-210e-469d-a005-3aee7e31c4c2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:44.330173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400538d0-210e-469d-a005-3aee7e31c4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:44.330442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400538d0-210e-469d-a005-3aee7e31c4c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:44.331167 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400538d0-210e-469d-a005-3aee7e31c4c2 None admin] RBAC: Adding query filter params (interface=admin, service_id=11febbccc7fc466ea0c7c846ad3e5066, region_id=RegionOne) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:44.332041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400538d0-210e-469d-a005-3aee7e31c4c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:44.333551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400538d0-210e-469d-a005-3aee7e31c4c2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:44.339824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 491/982] 10.222.0.26 () {56 vars in 1249 bytes} [Sat Sep 10 14:01:44 2022] GET /identity/v3/endpoints?service_id=11febbccc7fc466ea0c7c846ad3e5066&interface=admin®ion_id=RegionOne => generated 193 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:44.345695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400538d0-210e-469d-a005-3aee7e31c4c2 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:44.368816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:44.393541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247980c8-3e4b-42a5-8e7d-1c114b91047a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:44.456453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526f4fb-4ff8-4898-a725-868d16f43aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:44.457744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526f4fb-4ff8-4898-a725-868d16f43aaa None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:44.458016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526f4fb-4ff8-4898-a725-868d16f43aaa None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:44.458308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3526f4fb-4ff8-4898-a725-868d16f43aaa None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:44.458921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526f4fb-4ff8-4898-a725-868d16f43aaa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:44.461199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3526f4fb-4ff8-4898-a725-868d16f43aaa None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:44.477860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.sql.core [None req-3526f4fb-4ff8-4898-a725-868d16f43aaa None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 14:01:44.477860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 10 14:01:44.477860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [parameters: {'user_id': '6c8b8e0bedc64fc58b40c39971f5e787', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 10 14:01:44.477860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81588) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-3526f4fb-4ff8-4898-a725-868d16f43aaa None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application conn.query(q) Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result.read() Sep 10 14:01:44.484965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 14:01:44.485834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application next(self.gen) Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application yield resource Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application next(self.gen) Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.session.rollback() Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.force_reraise() Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise self.value Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application session.commit() Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 10 14:01:44.486820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._prepare_impl() Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.session.flush() Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._flush(objects) Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application compat.raise_( Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application flush_context.execute() Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rec.execute(self) Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 10 14:01:44.488420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 10 14:01:44.489337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result = self._query(query) Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application conn.query(q) Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application result.read() Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 10 14:01:44.490182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [parameters: {'user_id': '6c8b8e0bedc64fc58b40c39971f5e787', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:44.491013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 10 14:01:44.493691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 10 14:01:44.494899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 10 14:01:44.494899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 10 14:01:44.494899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 10 14:01:44.494899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 10 14:01:44.494899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 10 14:01:44.494899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:44.494899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 492/983] 10.222.0.26 () {60 vars in 1131 bytes} [Sat Sep 10 14:01:44 2022] POST /identity/v3/users => generated 160 bytes in 237 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:44.494899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526f4fb-4ff8-4898-a725-868d16f43aaa None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:44.509345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400538d0-210e-469d-a005-3aee7e31c4c2 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:44.548606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b23846-6928-4cc6-9163-503fd71208ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:44.549514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b23846-6928-4cc6-9163-503fd71208ce None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:44.549795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b23846-6928-4cc6-9163-503fd71208ce None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:44.550098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b23846-6928-4cc6-9163-503fd71208ce None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:44.550893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b23846-6928-4cc6-9163-503fd71208ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:44.552347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b23846-6928-4cc6-9163-503fd71208ce None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:44.558566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 492/984] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 10 14:01:44 2022] GET /identity/v3/services => generated 1497 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:01:44.560032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1bf5-6ce3-44cf-86bd-7af88532ac83 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:44.560472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1bf5-6ce3-44cf-86bd-7af88532ac83 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:44.560747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85d1bf5-6ce3-44cf-86bd-7af88532ac83 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:44.562139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 493/985] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:44.568321 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443631c6-4a2e-42b1-95bb-3f0fc5c28214 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:44.568699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443631c6-4a2e-42b1-95bb-3f0fc5c28214 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:44.568872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443631c6-4a2e-42b1-95bb-3f0fc5c28214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:44.589781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-443631c6-4a2e-42b1-95bb-3f0fc5c28214 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:44.591002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-443631c6-4a2e-42b1-95bb-3f0fc5c28214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:44.607726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:44.622103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3526f4fb-4ff8-4898-a725-868d16f43aaa None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:44.665151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e69d5-8a90-47b3-983e-9822573a0734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:44.666078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e69d5-8a90-47b3-983e-9822573a0734 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:44.666515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e69d5-8a90-47b3-983e-9822573a0734 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:44.667013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e69d5-8a90-47b3-983e-9822573a0734 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:44.675631 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 494/986] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:44.680887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e69d5-8a90-47b3-983e-9822573a0734 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:44.682942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72e69d5-8a90-47b3-983e-9822573a0734 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-f72e69d5-8a90-47b3-983e-9822573a0734 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:44.697162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:44.698225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:44.699298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:44.699298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:44.699298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:44.699298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:44.699298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:44.699298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:44.699298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:44.699298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:44.699298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 10 14:01:44.699298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:44.702062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 493/987] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:01:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:44.708691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443631c6-4a2e-42b1-95bb-3f0fc5c28214 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:44.819736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443631c6-4a2e-42b1-95bb-3f0fc5c28214 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:44.856322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3a2483-69a7-4e73-9bae-560c429db2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:44.856926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2483-69a7-4e73-9bae-560c429db2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:44.857197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2483-69a7-4e73-9bae-560c429db2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:44.857465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3a2483-69a7-4e73-9bae-560c429db2a9 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:44.858161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a2483-69a7-4e73-9bae-560c429db2a9 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:44.858459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a2483-69a7-4e73-9bae-560c429db2a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:44.859931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3a2483-69a7-4e73-9bae-560c429db2a9 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:44.877232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:44.898066 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 495/988] 10.222.0.26 () {56 vars in 1101 bytes} [Sat Sep 10 14:01:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:44.957184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e69d5-8a90-47b3-983e-9822573a0734 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:44.968600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:45.081650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e69d5-8a90-47b3-983e-9822573a0734 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:45.119073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0dbfc-d365-4d18-a62c-dd65df71d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:45.119725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0dbfc-d365-4d18-a62c-dd65df71d1e5 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:45.120012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0dbfc-d365-4d18-a62c-dd65df71d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:45.120288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0dbfc-d365-4d18-a62c-dd65df71d1e5 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:45.120986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0dbfc-d365-4d18-a62c-dd65df71d1e5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:45.122308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0dbfc-d365-4d18-a62c-dd65df71d1e5 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:45.142143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 494/989] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:01:44 2022] POST /identity/v3/services => generated 242 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:46.538809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe708-6a46-46e5-801e-71cb95b177f0 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:46.539346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe708-6a46-46e5-801e-71cb95b177f0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:46.539346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbe708-6a46-46e5-801e-71cb95b177f0 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:46.540735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 496/990] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:46.546268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1251d-5aff-4401-b478-9803f3576096 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:46.546647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1251d-5aff-4401-b478-9803f3576096 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:46.546818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff1251d-5aff-4401-b478-9803f3576096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:46.565280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-fff1251d-5aff-4401-b478-9803f3576096 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:46.566898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-fff1251d-5aff-4401-b478-9803f3576096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:46.625790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:46.641887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 495/991] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:46.835758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c25102-839d-4fc7-8f8d-0b2d92115df0 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:46.835947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c25102-839d-4fc7-8f8d-0b2d92115df0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:46.836209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c25102-839d-4fc7-8f8d-0b2d92115df0 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:46.837699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 497/992] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:46.842837 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60803389-5220-4935-8fac-30d641b290e3 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:46.843124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60803389-5220-4935-8fac-30d641b290e3 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:46.843395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60803389-5220-4935-8fac-30d641b290e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:46.862008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-60803389-5220-4935-8fac-30d641b290e3 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:46.863591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-60803389-5220-4935-8fac-30d641b290e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:46.910428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c25102-839d-4fc7-8f8d-0b2d92115df0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:46.934939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 496/993] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:47.015617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c25102-839d-4fc7-8f8d-0b2d92115df0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:47.050994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188b8ac-cc1e-4d18-879d-701d49f40cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:47.051910 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188b8ac-cc1e-4d18-879d-701d49f40cdb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:47.052338 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188b8ac-cc1e-4d18-879d-701d49f40cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:47.052759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188b8ac-cc1e-4d18-879d-701d49f40cdb None admin] PATH_INFO: `/v3/services/image` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:47.053495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188b8ac-cc1e-4d18-879d-701d49f40cdb None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:47.055015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188b8ac-cc1e-4d18-879d-701d49f40cdb None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-0188b8ac-cc1e-4d18-879d-701d49f40cdb None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:47.064083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:47.065271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 14:01:47.066339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:47.068515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 498/994] 10.222.0.26 () {56 vars in 1105 bytes} [Sat Sep 10 14:01:46 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:47.074351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60803389-5220-4935-8fac-30d641b290e3 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:47.172774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60803389-5220-4935-8fac-30d641b290e3 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:47.196334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:47.209455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d78205-fd13-4165-b155-4a49d2a2cd9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:47.210084 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d78205-fd13-4165-b155-4a49d2a2cd9d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:47.210360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d78205-fd13-4165-b155-4a49d2a2cd9d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:47.210628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d78205-fd13-4165-b155-4a49d2a2cd9d None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:47.211279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d78205-fd13-4165-b155-4a49d2a2cd9d None admin] RBAC: Adding query filter params (name=image) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:47.211609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d78205-fd13-4165-b155-4a49d2a2cd9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:47.213030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d78205-fd13-4165-b155-4a49d2a2cd9d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:47.214370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188b8ac-cc1e-4d18-879d-701d49f40cdb None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:47.226836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 497/995] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:47 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:47.228123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d78205-fd13-4165-b155-4a49d2a2cd9d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:47.279909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:47.322805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188b8ac-cc1e-4d18-879d-701d49f40cdb None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:47.337675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d78205-fd13-4165-b155-4a49d2a2cd9d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:47.357617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fd8d2-1831-4d6f-9310-6c2eeee52e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:47.358232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fd8d2-1831-4d6f-9310-6c2eeee52e5d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:47.358505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fd8d2-1831-4d6f-9310-6c2eeee52e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:47.358768 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fd8d2-1831-4d6f-9310-6c2eeee52e5d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:47.359339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fd8d2-1831-4d6f-9310-6c2eeee52e5d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:47.360734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fd8d2-1831-4d6f-9310-6c2eeee52e5d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-333fd8d2-1831-4d6f-9310-6c2eeee52e5d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:47.370041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:47.371232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 10 14:01:47.372416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:47.373168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76daac9a-b32c-42be-87b9-0ba190d7cc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:47.373259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 499/996] 10.222.0.26 () {56 vars in 1126 bytes} [Sat Sep 10 14:01:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:47.373331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daac9a-b32c-42be-87b9-0ba190d7cc97 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:47.373331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daac9a-b32c-42be-87b9-0ba190d7cc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:47.373331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76daac9a-b32c-42be-87b9-0ba190d7cc97 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:47.373513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f0b5d-6f00-42f8-b406-4c37b401261a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:47.373513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f0b5d-6f00-42f8-b406-4c37b401261a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:47.373513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f0b5d-6f00-42f8-b406-4c37b401261a None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:47.373689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76daac9a-b32c-42be-87b9-0ba190d7cc97 None admin] RBAC: Adding query filter params (type=image) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:47.373689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76daac9a-b32c-42be-87b9-0ba190d7cc97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:47.378723 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 500/997] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:47.379046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76daac9a-b32c-42be-87b9-0ba190d7cc97 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:47.379151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f0b5d-6f00-42f8-b406-4c37b401261a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:47.383341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 498/998] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:47 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:47.383341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0f47-0494-4800-b42b-5e75164c05cc None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:47.383341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0f47-0494-4800-b42b-5e75164c05cc None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:47.383341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb0f47-0494-4800-b42b-5e75164c05cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:47.405231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-ddbb0f47-0494-4800-b42b-5e75164c05cc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:47.406500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ddbb0f47-0494-4800-b42b-5e75164c05cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:47.461918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:47.486668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 499/999] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:47.510993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f0b5d-6f00-42f8-b406-4c37b401261a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:47.537906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:47.550334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e8da3-9bb9-4623-90a8-4fdd5a08b04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:47.551001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8da3-9bb9-4623-90a8-4fdd5a08b04f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:47.551263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8da3-9bb9-4623-90a8-4fdd5a08b04f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:47.551511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e8da3-9bb9-4623-90a8-4fdd5a08b04f None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:47.552155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e8da3-9bb9-4623-90a8-4fdd5a08b04f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:47.552455 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e8da3-9bb9-4623-90a8-4fdd5a08b04f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:47.553895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e8da3-9bb9-4623-90a8-4fdd5a08b04f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:47.559914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 501/1000] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 10 14:01:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:47.566494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0f47-0494-4800-b42b-5e75164c05cc None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:47.671521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb0f47-0494-4800-b42b-5e75164c05cc None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:47.704766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30567e-6864-45b8-8561-50cd79331f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:47.705286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30567e-6864-45b8-8561-50cd79331f8a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:47.705561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30567e-6864-45b8-8561-50cd79331f8a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:47.705827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef30567e-6864-45b8-8561-50cd79331f8a None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:47.707039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30567e-6864-45b8-8561-50cd79331f8a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:47.707338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30567e-6864-45b8-8561-50cd79331f8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:47.708739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef30567e-6864-45b8-8561-50cd79331f8a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:47.718262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 500/1001] 10.222.0.26 () {56 vars in 1122 bytes} [Sat Sep 10 14:01:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:47.719834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e8da3-9bb9-4623-90a8-4fdd5a08b04f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:47.763932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30567e-6864-45b8-8561-50cd79331f8a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:47.812562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:47.823732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e8da3-9bb9-4623-90a8-4fdd5a08b04f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:47.857486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4426-b7dc-415f-840b-faa2e785d36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:47.858096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4426-b7dc-415f-840b-faa2e785d36a None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:47.858367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4426-b7dc-415f-840b-faa2e785d36a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:47.858635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2e4426-b7dc-415f-840b-faa2e785d36a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:47.859306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e4426-b7dc-415f-840b-faa2e785d36a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:47.860707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2e4426-b7dc-415f-840b-faa2e785d36a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:47.875103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef30567e-6864-45b8-8561-50cd79331f8a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:47.878080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 502/1002] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:01:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:47.927527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ef76a-6f1f-4279-b0b8-d30c177fc411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:47.928532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ef76a-6f1f-4279-b0b8-d30c177fc411 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:47.928986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ef76a-6f1f-4279-b0b8-d30c177fc411 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:47.929460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9ef76a-6f1f-4279-b0b8-d30c177fc411 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:47.930230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ef76a-6f1f-4279-b0b8-d30c177fc411 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:47.931928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9ef76a-6f1f-4279-b0b8-d30c177fc411 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-0d9ef76a-6f1f-4279-b0b8-d30c177fc411 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:47.942175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:47.943263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 14:01:47.944140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:47.944898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 501/1003] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:47.951013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4426-b7dc-415f-840b-faa2e785d36a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:48.006350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:48.067767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2e4426-b7dc-415f-840b-faa2e785d36a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:48.103491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b41add-4000-4b66-b385-e46697931cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:48.104110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b41add-4000-4b66-b385-e46697931cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:48.104385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b41add-4000-4b66-b385-e46697931cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:48.104653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b41add-4000-4b66-b385-e46697931cc3 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:48.105286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b41add-4000-4b66-b385-e46697931cc3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:48.105585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b41add-4000-4b66-b385-e46697931cc3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:48.106962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b41add-4000-4b66-b385-e46697931cc3 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:48.111881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 503/1004] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:48.118421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ef76a-6f1f-4279-b0b8-d30c177fc411 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:48.224815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9ef76a-6f1f-4279-b0b8-d30c177fc411 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:48.227229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:48.261749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c6747-4d46-4778-9bee-e3e7ee764045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:48.262276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c6747-4d46-4778-9bee-e3e7ee764045 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:48.262556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c6747-4d46-4778-9bee-e3e7ee764045 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:48.262827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c6747-4d46-4778-9bee-e3e7ee764045 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:48.263477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c6747-4d46-4778-9bee-e3e7ee764045 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:48.263783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c6747-4d46-4778-9bee-e3e7ee764045 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:48.265157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c6747-4d46-4778-9bee-e3e7ee764045 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:48.272564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 502/1005] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:48.277495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b41add-4000-4b66-b385-e46697931cc3 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:48.386098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b41add-4000-4b66-b385-e46697931cc3 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:48.399425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:48.423478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf641033-3e28-49dd-8cbe-7232531aec93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:48.423478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf641033-3e28-49dd-8cbe-7232531aec93 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:48.423478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf641033-3e28-49dd-8cbe-7232531aec93 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:48.423819 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf641033-3e28-49dd-8cbe-7232531aec93 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:48.426688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf641033-3e28-49dd-8cbe-7232531aec93 None admin] RBAC: Adding query filter params (interface=public, service_id=3042871eb6024ee3ac31c7edb7e1b808, region_id=RegionOne) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:48.426688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf641033-3e28-49dd-8cbe-7232531aec93 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:48.426877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf641033-3e28-49dd-8cbe-7232531aec93 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:48.438162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 504/1006] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 10 14:01:48 2022] GET /identity/v3/endpoints?service_id=3042871eb6024ee3ac31c7edb7e1b808&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:48.439582 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c6747-4d46-4778-9bee-e3e7ee764045 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:48.543759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c6747-4d46-4778-9bee-e3e7ee764045 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:48.577675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba94bc-f594-405a-9aa5-c4675b162579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:48.578330 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba94bc-f594-405a-9aa5-c4675b162579 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:48.578577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba94bc-f594-405a-9aa5-c4675b162579 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:48.578820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba94bc-f594-405a-9aa5-c4675b162579 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:48.579420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba94bc-f594-405a-9aa5-c4675b162579 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:48.581009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba94bc-f594-405a-9aa5-c4675b162579 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:48.591641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 503/1007] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 10 14:01:48 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:01:49.569412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60235b-cd41-4c6b-87e0-418ac75bbdec None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:49.569925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60235b-cd41-4c6b-87e0-418ac75bbdec None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:49.570002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60235b-cd41-4c6b-87e0-418ac75bbdec None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:49.571437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 505/1008] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:49.576467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb700f-9def-4e92-98ae-ec086a0e4760 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:49.576754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb700f-9def-4e92-98ae-ec086a0e4760 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:49.577027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb700f-9def-4e92-98ae-ec086a0e4760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:49.595926 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-00fb700f-9def-4e92-98ae-ec086a0e4760 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:49.597654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-00fb700f-9def-4e92-98ae-ec086a0e4760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:49.623012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:49.664896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 504/1009] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:49 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:49.932111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60235b-cd41-4c6b-87e0-418ac75bbdec None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:50.039285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60235b-cd41-4c6b-87e0-418ac75bbdec None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:50.061201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbb632-2c4a-47e3-a085-cdea181090e4 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:50.061531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbb632-2c4a-47e3-a085-cdea181090e4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:50.061762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbbb632-2c4a-47e3-a085-cdea181090e4 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:50.063114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 505/1010] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:01:50.068702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ef9fc-98ea-4665-9f4a-efa3d355fac7 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:50.068981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ef9fc-98ea-4665-9f4a-efa3d355fac7 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:50.069251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454ef9fc-98ea-4665-9f4a-efa3d355fac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:50.074882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc199ea5-9ec4-432c-8462-05b62bd28448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:50.075506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc199ea5-9ec4-432c-8462-05b62bd28448 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:50.075789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc199ea5-9ec4-432c-8462-05b62bd28448 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:50.076055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc199ea5-9ec4-432c-8462-05b62bd28448 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:50.076697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc199ea5-9ec4-432c-8462-05b62bd28448 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:50.078064 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc199ea5-9ec4-432c-8462-05b62bd28448 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:50.088007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-454ef9fc-98ea-4665-9f4a-efa3d355fac7 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:50.089194 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-454ef9fc-98ea-4665-9f4a-efa3d355fac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:50.094754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 506/1011] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:01:49 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:50.172100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 506/1012] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:50 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:50.443318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc199ea5-9ec4-432c-8462-05b62bd28448 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:50.479752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:50.548752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc199ea5-9ec4-432c-8462-05b62bd28448 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:50.582980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a22fd-cd0d-4bd0-8cef-554a17e0c51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:50.583670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a22fd-cd0d-4bd0-8cef-554a17e0c51a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:50.583904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a22fd-cd0d-4bd0-8cef-554a17e0c51a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:50.584163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a22fd-cd0d-4bd0-8cef-554a17e0c51a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:50.586185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a22fd-cd0d-4bd0-8cef-554a17e0c51a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:50.587146 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a22fd-cd0d-4bd0-8cef-554a17e0c51a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:50.588912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 507/1013] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:50.594420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ef9fc-98ea-4665-9f4a-efa3d355fac7 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:50.683905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:50.699508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454ef9fc-98ea-4665-9f4a-efa3d355fac7 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:50.734484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3447c0d-c7a7-4e97-a97e-2e6e6d0a902c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:50.735085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3447c0d-c7a7-4e97-a97e-2e6e6d0a902c None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:50.735357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3447c0d-c7a7-4e97-a97e-2e6e6d0a902c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:50.735649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3447c0d-c7a7-4e97-a97e-2e6e6d0a902c None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:50.736301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3447c0d-c7a7-4e97-a97e-2e6e6d0a902c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:50.738330 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3447c0d-c7a7-4e97-a97e-2e6e6d0a902c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:50.771468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 507/1014] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:01:50 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:50.803037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daf15c-44c8-42a5-8bde-c9e78c95ad78 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:50.803510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daf15c-44c8-42a5-8bde-c9e78c95ad78 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:50.803862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daf15c-44c8-42a5-8bde-c9e78c95ad78 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:50.805343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 508/1015] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:50.812655 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32461254-10be-40ed-8c50-a076f5d974cf None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:50.812655 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32461254-10be-40ed-8c50-a076f5d974cf None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:50.812655 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32461254-10be-40ed-8c50-a076f5d974cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:50.834126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-32461254-10be-40ed-8c50-a076f5d974cf None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:50.835357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-32461254-10be-40ed-8c50-a076f5d974cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:50.910727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 508/1016] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:50 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:51.187254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daf15c-44c8-42a5-8bde-c9e78c95ad78 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:51.292828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daf15c-44c8-42a5-8bde-c9e78c95ad78 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:51.295148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:51.330800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e6f8b-67ce-4eca-ba5f-030387dd1860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:51.332006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e6f8b-67ce-4eca-ba5f-030387dd1860 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:51.332285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e6f8b-67ce-4eca-ba5f-030387dd1860 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:51.332602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e6f8b-67ce-4eca-ba5f-030387dd1860 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:51.333137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e6f8b-67ce-4eca-ba5f-030387dd1860 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:51.334511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e6f8b-67ce-4eca-ba5f-030387dd1860 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-712e6f8b-67ce-4eca-ba5f-030387dd1860 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:51.343459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:51.344449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:51.345281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 509/1017] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:51.351251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32461254-10be-40ed-8c50-a076f5d974cf None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:51.422012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:51.460677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32461254-10be-40ed-8c50-a076f5d974cf None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:51.494167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c20ae-a73c-4b7c-8999-835bba3d9232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:51.494769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c20ae-a73c-4b7c-8999-835bba3d9232 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:51.495045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c20ae-a73c-4b7c-8999-835bba3d9232 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:51.495314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c20ae-a73c-4b7c-8999-835bba3d9232 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:51.495969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c20ae-a73c-4b7c-8999-835bba3d9232 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:51.496267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c20ae-a73c-4b7c-8999-835bba3d9232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:51.497630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c20ae-a73c-4b7c-8999-835bba3d9232 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:51.508037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 509/1018] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:51.509515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e6f8b-67ce-4eca-ba5f-030387dd1860 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:51.614194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e6f8b-67ce-4eca-ba5f-030387dd1860 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:51.649078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f3896-7b0d-46df-a373-1b79ac64bbf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:51.650011 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3896-7b0d-46df-a373-1b79ac64bbf2 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:51.650486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3896-7b0d-46df-a373-1b79ac64bbf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:51.650951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f3896-7b0d-46df-a373-1b79ac64bbf2 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:51.651812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f3896-7b0d-46df-a373-1b79ac64bbf2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:51.652307 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f3896-7b0d-46df-a373-1b79ac64bbf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:51.653880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f3896-7b0d-46df-a373-1b79ac64bbf2 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:51.663627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 510/1019] 10.222.0.26 () {56 vars in 1112 bytes} [Sat Sep 10 14:01:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:51.665505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c20ae-a73c-4b7c-8999-835bba3d9232 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:51.769347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c20ae-a73c-4b7c-8999-835bba3d9232 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:51.803991 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffba07af-8b77-44d0-a4a0-bd404dc82128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:51.804595 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba07af-8b77-44d0-a4a0-bd404dc82128 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:51.804872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba07af-8b77-44d0-a4a0-bd404dc82128 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:51.805141 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffba07af-8b77-44d0-a4a0-bd404dc82128 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:51.805808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba07af-8b77-44d0-a4a0-bd404dc82128 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:51.807166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffba07af-8b77-44d0-a4a0-bd404dc82128 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:51.811688 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:51.835478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 510/1020] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:01:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:52.294245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62991c47-7c43-478f-a410-13773c6c1584 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:52.294600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62991c47-7c43-478f-a410-13773c6c1584 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:52.294790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62991c47-7c43-478f-a410-13773c6c1584 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:52.296206 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 511/1021] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:52.301593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a895e89-ea70-4d15-9788-973e4999cd4b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:52.301970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a895e89-ea70-4d15-9788-973e4999cd4b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:52.302352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a895e89-ea70-4d15-9788-973e4999cd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:52.321256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-1a895e89-ea70-4d15-9788-973e4999cd4b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:52.322437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1a895e89-ea70-4d15-9788-973e4999cd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:52.399507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 511/1022] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:52 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:52.666244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62991c47-7c43-478f-a410-13773c6c1584 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:52.698136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:52.775085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62991c47-7c43-478f-a410-13773c6c1584 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:52.809693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c7c44-0a45-45a1-bcd0-17fa34c5ebc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:52.810347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c7c44-0a45-45a1-bcd0-17fa34c5ebc1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:52.810624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c7c44-0a45-45a1-bcd0-17fa34c5ebc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:52.810902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5c7c44-0a45-45a1-bcd0-17fa34c5ebc1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:52.811527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c7c44-0a45-45a1-bcd0-17fa34c5ebc1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:52.812947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5c7c44-0a45-45a1-bcd0-17fa34c5ebc1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-bb5c7c44-0a45-45a1-bcd0-17fa34c5ebc1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:52.823464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:52.824541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 14:01:52.825361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:52.828956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 512/1023] 10.222.0.26 () {56 vars in 1105 bytes} [Sat Sep 10 14:01:52 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:52.832254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a895e89-ea70-4d15-9788-973e4999cd4b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:52.933937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a895e89-ea70-4d15-9788-973e4999cd4b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:52.951164 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:52.971064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6d9e8-fbcf-4ec8-a695-150f2e1e044d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:52.972292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6d9e8-fbcf-4ec8-a695-150f2e1e044d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:52.972719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6d9e8-fbcf-4ec8-a695-150f2e1e044d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:52.972986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6d9e8-fbcf-4ec8-a695-150f2e1e044d None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:52.973610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6d9e8-fbcf-4ec8-a695-150f2e1e044d None admin] RBAC: Adding query filter params (name=image) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:52.973908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6d9e8-fbcf-4ec8-a695-150f2e1e044d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:52.975272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6d9e8-fbcf-4ec8-a695-150f2e1e044d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:52.984804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 512/1024] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:52 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:52.986140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c7c44-0a45-45a1-bcd0-17fa34c5ebc1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:53.013289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07a1f-16ac-4297-868f-c2a4c34de10e None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:53.013567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07a1f-16ac-4297-868f-c2a4c34de10e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:53.013836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b07a1f-16ac-4297-868f-c2a4c34de10e None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:53.015155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 513/1025] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:01:53.020582 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6ded3-f8e9-4d7f-9ffd-e953f5a14833 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:53.020856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6ded3-f8e9-4d7f-9ffd-e953f5a14833 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:53.021128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad6ded3-f8e9-4d7f-9ffd-e953f5a14833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:53.041952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-0ad6ded3-f8e9-4d7f-9ffd-e953f5a14833 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:53.043143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0ad6ded3-f8e9-4d7f-9ffd-e953f5a14833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:53.096048 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:53.103209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5c7c44-0a45-45a1-bcd0-17fa34c5ebc1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:53.119487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 514/1026] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:53.147141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa0b1f-ccb9-4af4-9672-3afac5b91f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:53.147769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa0b1f-ccb9-4af4-9672-3afac5b91f60 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:53.148050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa0b1f-ccb9-4af4-9672-3afac5b91f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:53.148321 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aa0b1f-ccb9-4af4-9672-3afac5b91f60 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:53.148946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa0b1f-ccb9-4af4-9672-3afac5b91f60 None admin] RBAC: Adding query filter params (type=image) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:53.149245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa0b1f-ccb9-4af4-9672-3afac5b91f60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:53.150619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6aa0b1f-ccb9-4af4-9672-3afac5b91f60 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:53.160612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 513/1027] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:52 2022] GET /identity/v3/services?type=image => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:53.161756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6ded3-f8e9-4d7f-9ffd-e953f5a14833 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:53.262838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad6ded3-f8e9-4d7f-9ffd-e953f5a14833 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:53.273862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:53.299410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f3474-4216-4f13-ac26-a522c8885baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:53.300064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f3474-4216-4f13-ac26-a522c8885baa None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:53.300338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f3474-4216-4f13-ac26-a522c8885baa None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:53.300605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2f3474-4216-4f13-ac26-a522c8885baa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:53.301296 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f3474-4216-4f13-ac26-a522c8885baa None admin] RBAC: Adding query filter params (interface=public, service_id=8d348fe8991b4e04a39f276ca0d15554, region_id=RegionOne) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:53.301600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f3474-4216-4f13-ac26-a522c8885baa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:53.302954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2f3474-4216-4f13-ac26-a522c8885baa None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:53.312629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 515/1028] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 10 14:01:53 2022] GET /identity/v3/endpoints?service_id=8d348fe8991b4e04a39f276ca0d15554&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:53.313642 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa0b1f-ccb9-4af4-9672-3afac5b91f60 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:53.394643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f3474-4216-4f13-ac26-a522c8885baa None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:53.415062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6aa0b1f-ccb9-4af4-9672-3afac5b91f60 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:53.462078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c360ed4-0608-4e04-8953-484351bfad9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:53.462680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c360ed4-0608-4e04-8953-484351bfad9e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:53.462959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c360ed4-0608-4e04-8953-484351bfad9e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:53.463225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c360ed4-0608-4e04-8953-484351bfad9e None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:53.463887 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c360ed4-0608-4e04-8953-484351bfad9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:53.465258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c360ed4-0608-4e04-8953-484351bfad9e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:53.472837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 514/1029] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 10 14:01:53 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:01:53.514411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2f3474-4216-4f13-ac26-a522c8885baa None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:53.553949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90576a3-4f44-421b-a099-a510d422fce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:53.554599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90576a3-4f44-421b-a099-a510d422fce5 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:53.554949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90576a3-4f44-421b-a099-a510d422fce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:53.555229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90576a3-4f44-421b-a099-a510d422fce5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:53.556422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90576a3-4f44-421b-a099-a510d422fce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:53.557886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90576a3-4f44-421b-a099-a510d422fce5 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:53.563603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 516/1030] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:53.565686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c360ed4-0608-4e04-8953-484351bfad9e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:53.594218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:53.678215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c360ed4-0608-4e04-8953-484351bfad9e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:53.712579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88764f4f-5a44-4e43-99fb-ef7c3c0b8eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:53.713174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88764f4f-5a44-4e43-99fb-ef7c3c0b8eed None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:53.713458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88764f4f-5a44-4e43-99fb-ef7c3c0b8eed None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:53.713725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88764f4f-5a44-4e43-99fb-ef7c3c0b8eed None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:53.714366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88764f4f-5a44-4e43-99fb-ef7c3c0b8eed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:53.715822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88764f4f-5a44-4e43-99fb-ef7c3c0b8eed None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:53.739239 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 515/1031] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:01:53 2022] POST /identity/v3/users => generated 285 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:54.113074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d902b2b-f31a-4cf7-be08-3af2a21fcd2f None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:54.113640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d902b2b-f31a-4cf7-be08-3af2a21fcd2f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:54.113640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d902b2b-f31a-4cf7-be08-3af2a21fcd2f None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:54.115232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 517/1032] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:54.120662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a3fb9-112e-4edb-b803-a51c3b5ac9eb None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:54.120950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a3fb9-112e-4edb-b803-a51c3b5ac9eb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:54.121221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a3fb9-112e-4edb-b803-a51c3b5ac9eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:54.139885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-886a3fb9-112e-4edb-b803-a51c3b5ac9eb None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:54.141060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-886a3fb9-112e-4edb-b803-a51c3b5ac9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:54.209880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 516/1033] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:54.479283 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d902b2b-f31a-4cf7-be08-3af2a21fcd2f None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:54.526311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:54.584464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d902b2b-f31a-4cf7-be08-3af2a21fcd2f None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:54.618216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d7303-f6b8-4913-8dc8-4c125a9aa89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:54.618840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d7303-f6b8-4913-8dc8-4c125a9aa89f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:54.619115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d7303-f6b8-4913-8dc8-4c125a9aa89f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:54.619385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8d7303-f6b8-4913-8dc8-4c125a9aa89f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:54.629860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8d7303-f6b8-4913-8dc8-4c125a9aa89f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-2b8d7303-f6b8-4913-8dc8-4c125a9aa89f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:54.634617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:54.635767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:01:54.636611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:54.637811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 518/1034] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 10 14:01:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:54.641622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a3fb9-112e-4edb-b803-a51c3b5ac9eb None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:54.653181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:54.745192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a3fb9-112e-4edb-b803-a51c3b5ac9eb None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:54.779536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bc49f-cf56-40e6-872b-c28b282f5e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:54.780145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc49f-cf56-40e6-872b-c28b282f5e05 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:54.780414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc49f-cf56-40e6-872b-c28b282f5e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:54.780678 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048bc49f-cf56-40e6-872b-c28b282f5e05 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:54.781338 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bc49f-cf56-40e6-872b-c28b282f5e05 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:54.781654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bc49f-cf56-40e6-872b-c28b282f5e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:54.783015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048bc49f-cf56-40e6-872b-c28b282f5e05 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:54.796240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 517/1035] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 10 14:01:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:54.797260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d7303-f6b8-4913-8dc8-4c125a9aa89f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:54.898685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8d7303-f6b8-4913-8dc8-4c125a9aa89f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:54.900207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:54.935352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ca1cc-1213-4223-9b2d-7576be73b5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:54.936000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca1cc-1213-4223-9b2d-7576be73b5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:54.936271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca1cc-1213-4223-9b2d-7576be73b5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:54.936534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59ca1cc-1213-4223-9b2d-7576be73b5a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:54.946919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ca1cc-1213-4223-9b2d-7576be73b5a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:54.948385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59ca1cc-1213-4223-9b2d-7576be73b5a2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-e59ca1cc-1213-4223-9b2d-7576be73b5a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:54.962483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:54.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:54.968795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:54.968795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:54.968795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:54.968795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 519/1036] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:54.971280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bc49f-cf56-40e6-872b-c28b282f5e05 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:55.052665 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:55.084112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048bc49f-cf56-40e6-872b-c28b282f5e05 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:55.119317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c7ba4-d2d0-45e1-b1ba-c34bfb1b20dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:55.119945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c7ba4-d2d0-45e1-b1ba-c34bfb1b20dc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:55.120218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c7ba4-d2d0-45e1-b1ba-c34bfb1b20dc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:55.120485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2c7ba4-d2d0-45e1-b1ba-c34bfb1b20dc None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:55.121121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c7ba4-d2d0-45e1-b1ba-c34bfb1b20dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:55.121417 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c7ba4-d2d0-45e1-b1ba-c34bfb1b20dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:55.122788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2c7ba4-d2d0-45e1-b1ba-c34bfb1b20dc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:55.140226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 518/1037] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:55.141863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ca1cc-1213-4223-9b2d-7576be73b5a2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:55.244388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59ca1cc-1213-4223-9b2d-7576be73b5a2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:55.278649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5fbcb-4f17-4113-a738-78653b300605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:55.279616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5fbcb-4f17-4113-a738-78653b300605 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:55.280071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5fbcb-4f17-4113-a738-78653b300605 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:55.280515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e5fbcb-4f17-4113-a738-78653b300605 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:55.289115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5fbcb-4f17-4113-a738-78653b300605 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:55.290748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e5fbcb-4f17-4113-a738-78653b300605 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-f9e5fbcb-4f17-4113-a738-78653b300605 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:01:55.303014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:55.304032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:55.304974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:55.304974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:55.304974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:01:55.304974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:01:55.304974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:01:55.304974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:01:55.304974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:01:55.304974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:01:55.304974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:01:55.304974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:55.307521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 520/1038] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 10 14:01:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:55.311274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c7ba4-d2d0-45e1-b1ba-c34bfb1b20dc None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:55.412854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2c7ba4-d2d0-45e1-b1ba-c34bfb1b20dc None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:55.447760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:55.453612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7089f24a-2178-46a8-9d1a-8b51416ffdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:55.454216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089f24a-2178-46a8-9d1a-8b51416ffdae None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:55.454485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089f24a-2178-46a8-9d1a-8b51416ffdae None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:55.454748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7089f24a-2178-46a8-9d1a-8b51416ffdae None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:55.455492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089f24a-2178-46a8-9d1a-8b51416ffdae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:55.455794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089f24a-2178-46a8-9d1a-8b51416ffdae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:55.458715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7089f24a-2178-46a8-9d1a-8b51416ffdae None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:55.477542 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 519/1039] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 10 14:01:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:55.483136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5fbcb-4f17-4113-a738-78653b300605 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:55.500470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:55.593799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e5fbcb-4f17-4113-a738-78653b300605 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:55.627712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf3605-8a15-40ae-b90b-ef5f0a705596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:55.628958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf3605-8a15-40ae-b90b-ef5f0a705596 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:55.629224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf3605-8a15-40ae-b90b-ef5f0a705596 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:55.629485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf3605-8a15-40ae-b90b-ef5f0a705596 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:55.657605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6caa4-bd2b-4bbb-8bb5-719eebf35828 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:55.657920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6caa4-bd2b-4bbb-8bb5-719eebf35828 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:55.658154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde6caa4-bd2b-4bbb-8bb5-719eebf35828 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:55.659637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 520/1040] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:55.664504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f23a3-f11e-441b-85e7-5b5807846584 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:55.664783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f23a3-f11e-441b-85e7-5b5807846584 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:55.665052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f23a3-f11e-441b-85e7-5b5807846584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:55.683114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3b4f23a3-f11e-441b-85e7-5b5807846584 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:55.684318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3b4f23a3-f11e-441b-85e7-5b5807846584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:55.698942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf3605-8a15-40ae-b90b-ef5f0a705596 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:55.700477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf3605-8a15-40ae-b90b-ef5f0a705596 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:55.716413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:55.758129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 521/1041] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:55.806593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 521/1042] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:55 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:01:55.812351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f23a3-f11e-441b-85e7-5b5807846584 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:55.927253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f23a3-f11e-441b-85e7-5b5807846584 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:55.948293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:55.963872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e46fa-0129-4665-ab2f-adf5409bb766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:55.964503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e46fa-0129-4665-ab2f-adf5409bb766 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:55.964775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e46fa-0129-4665-ab2f-adf5409bb766 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:55.965040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157e46fa-0129-4665-ab2f-adf5409bb766 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:55.976809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e46fa-0129-4665-ab2f-adf5409bb766 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:55.978246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157e46fa-0129-4665-ab2f-adf5409bb766 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:55.993801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf9d69-300d-43da-a8b9-6ba44de910c1 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-157e46fa-0129-4665-ab2f-adf5409bb766 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:55.993964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:55.994850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:55.995769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:55.995769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:55.995769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:55.995942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf9d69-300d-43da-a8b9-6ba44de910c1 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:55.995942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaf9d69-300d-43da-a8b9-6ba44de910c1 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:55.996073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 522/1043] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:56.000156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 522/1044] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:01:56.000745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e46fa-0129-4665-ab2f-adf5409bb766 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.006051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d1fa3-3af3-4a2f-8110-ce6945569bd7 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.006333 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d1fa3-3af3-4a2f-8110-ce6945569bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.006602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d1fa3-3af3-4a2f-8110-ce6945569bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.025639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e15d1fa3-3af3-4a2f-8110-ce6945569bd7 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:56.026833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e15d1fa3-3af3-4a2f-8110-ce6945569bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:56.094709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 523/1045] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:01:56.096226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d1fa3-3af3-4a2f-8110-ce6945569bd7 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.111718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157e46fa-0129-4665-ab2f-adf5409bb766 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.147319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:56.148076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcc9bf-c1c3-422e-8eb4-ae8af4c03ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.149059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcc9bf-c1c3-422e-8eb4-ae8af4c03ede None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.149059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcc9bf-c1c3-422e-8eb4-ae8af4c03ede None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.149474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bcc9bf-c1c3-422e-8eb4-ae8af4c03ede None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.149942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcc9bf-c1c3-422e-8eb4-ae8af4c03ede None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:56.150150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcc9bf-c1c3-422e-8eb4-ae8af4c03ede None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:56.151544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bcc9bf-c1c3-422e-8eb4-ae8af4c03ede None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:56.163363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 523/1046] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:56.168885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcc9bf-c1c3-422e-8eb4-ae8af4c03ede None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.205428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:56.215932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d1fa3-3af3-4a2f-8110-ce6945569bd7 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.254869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce279-52cc-49e3-9ff4-d5865bc2148d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.255799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce279-52cc-49e3-9ff4-d5865bc2148d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.256260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce279-52cc-49e3-9ff4-d5865bc2148d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.256727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce279-52cc-49e3-9ff4-d5865bc2148d None admin] PATH_INFO: `/v3/services/image` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.257500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ce279-52cc-49e3-9ff4-d5865bc2148d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:56.258881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6ce279-52cc-49e3-9ff4-d5865bc2148d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-5d6ce279-52cc-49e3-9ff4-d5865bc2148d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:56.267668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:56.268741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:56.269602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 524/1047] 10.222.0.26 () {56 vars in 1105 bytes} [Sat Sep 10 14:01:56 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:56.274757 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce279-52cc-49e3-9ff4-d5865bc2148d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.284036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bcc9bf-c1c3-422e-8eb4-ae8af4c03ede None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.322092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7d3d4-d367-4005-af67-7be1490b1c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.322709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7d3d4-d367-4005-af67-7be1490b1c45 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.322985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7d3d4-d367-4005-af67-7be1490b1c45 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.323252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea7d3d4-d367-4005-af67-7be1490b1c45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.329909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7d3d4-d367-4005-af67-7be1490b1c45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:56.332357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea7d3d4-d367-4005-af67-7be1490b1c45 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-7ea7d3d4-d367-4005-af67-7be1490b1c45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:01:56.343828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:56.344738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:56.346431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:01:56.346431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:01:56.346431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:01:56.346431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:01:56.346431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:01:56.346431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:01:56.346431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:01:56.346431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:56.346431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 524/1048] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:01:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:56.351614 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7d3d4-d367-4005-af67-7be1490b1c45 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.380570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce279-52cc-49e3-9ff4-d5865bc2148d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.389635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:56.416149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2767f7c-236a-47ac-ae73-6e6029ba0d67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.416746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2767f7c-236a-47ac-ae73-6e6029ba0d67 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.417020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2767f7c-236a-47ac-ae73-6e6029ba0d67 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.417291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2767f7c-236a-47ac-ae73-6e6029ba0d67 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.417911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2767f7c-236a-47ac-ae73-6e6029ba0d67 None admin] RBAC: Adding query filter params (name=image) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:56.418207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2767f7c-236a-47ac-ae73-6e6029ba0d67 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:56.419611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2767f7c-236a-47ac-ae73-6e6029ba0d67 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:56.425303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 525/1049] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:56.428349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2767f7c-236a-47ac-ae73-6e6029ba0d67 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.447660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:56.457758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea7d3d4-d367-4005-af67-7be1490b1c45 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.491622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df1dae-e011-4140-8d18-0a9838c452de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.492226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df1dae-e011-4140-8d18-0a9838c452de None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.492510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df1dae-e011-4140-8d18-0a9838c452de None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.492780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29df1dae-e011-4140-8d18-0a9838c452de None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.493465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df1dae-e011-4140-8d18-0a9838c452de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:56.493769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df1dae-e011-4140-8d18-0a9838c452de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:56.495203 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29df1dae-e011-4140-8d18-0a9838c452de None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:56.507405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 525/1050] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:01:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:56.508219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df1dae-e011-4140-8d18-0a9838c452de None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.529442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2767f7c-236a-47ac-ae73-6e6029ba0d67 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.567007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14e61d-2be4-4b11-b0dc-2fadb6917436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.567670 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14e61d-2be4-4b11-b0dc-2fadb6917436 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.567947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14e61d-2be4-4b11-b0dc-2fadb6917436 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.568211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e14e61d-2be4-4b11-b0dc-2fadb6917436 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.578772 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e14e61d-2be4-4b11-b0dc-2fadb6917436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-0e14e61d-2be4-4b11-b0dc-2fadb6917436 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:56.584278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:56.585295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 526/1051] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:01:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:01:56.586218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14e61d-2be4-4b11-b0dc-2fadb6917436 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.609772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29df1dae-e011-4140-8d18-0a9838c452de None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.616257 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:56.645596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec65ce-56f1-4741-81a9-de9a7568ee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.646216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec65ce-56f1-4741-81a9-de9a7568ee62 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.646488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec65ce-56f1-4741-81a9-de9a7568ee62 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.646749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec65ce-56f1-4741-81a9-de9a7568ee62 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.647370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec65ce-56f1-4741-81a9-de9a7568ee62 None admin] RBAC: Adding query filter params (type=image) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:56.647694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec65ce-56f1-4741-81a9-de9a7568ee62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:56.649053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffec65ce-56f1-4741-81a9-de9a7568ee62 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:56.653705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 526/1052] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:56.654461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec65ce-56f1-4741-81a9-de9a7568ee62 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.672106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:56.689975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e14e61d-2be4-4b11-b0dc-2fadb6917436 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.723568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b01cb-cc5c-4690-afb9-56c395d65bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.724091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b01cb-cc5c-4690-afb9-56c395d65bac None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.725042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b01cb-cc5c-4690-afb9-56c395d65bac None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.725042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128b01cb-cc5c-4690-afb9-56c395d65bac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.725323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b01cb-cc5c-4690-afb9-56c395d65bac None admin] RBAC: Adding query filter params (role.id=823d0de8de634181af75dabe48e834bb, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=cb24a1b55ee7408ab24c68991aec54ec) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:56.725625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b01cb-cc5c-4690-afb9-56c395d65bac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:56.727039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128b01cb-cc5c-4690-afb9-56c395d65bac None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:56.743476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 527/1053] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:56 2022] GET /identity/v3/role_assignments?user.id=cb24a1b55ee7408ab24c68991aec54ec&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=823d0de8de634181af75dabe48e834bb => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:56.743476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b01cb-cc5c-4690-afb9-56c395d65bac None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.756296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec65ce-56f1-4741-81a9-de9a7568ee62 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.796563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eff2f7-89d7-4364-9d9d-efc522933db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.797242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eff2f7-89d7-4364-9d9d-efc522933db8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.797540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eff2f7-89d7-4364-9d9d-efc522933db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.797716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eff2f7-89d7-4364-9d9d-efc522933db8 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.798435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eff2f7-89d7-4364-9d9d-efc522933db8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:56.798736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eff2f7-89d7-4364-9d9d-efc522933db8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:56.800495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eff2f7-89d7-4364-9d9d-efc522933db8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:56.810981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 527/1054] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:01:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:56.818629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eff2f7-89d7-4364-9d9d-efc522933db8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:56.869987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:56.877893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128b01cb-cc5c-4690-afb9-56c395d65bac None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.905953 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:56.918089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bf413-564a-4cdb-8185-40413743f215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.918699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bf413-564a-4cdb-8185-40413743f215 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.918973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bf413-564a-4cdb-8185-40413743f215 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.919247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bf413-564a-4cdb-8185-40413743f215 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:56.919947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bf413-564a-4cdb-8185-40413743f215 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:56.921303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02bf413-564a-4cdb-8185-40413743f215 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:56.938419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eff2f7-89d7-4364-9d9d-efc522933db8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:56.955833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 528/1055] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:01:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:01:56.985412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb2bab-6315-4c3c-afde-1bdf6e8a488a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:56.985965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb2bab-6315-4c3c-afde-1bdf6e8a488a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:56.986235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb2bab-6315-4c3c-afde-1bdf6e8a488a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:56.986499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb2bab-6315-4c3c-afde-1bdf6e8a488a None admin] PATH_INFO: `/v3/users/9effc53793ed40008f1d65d7831cd85e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:57.002742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb2bab-6315-4c3c-afde-1bdf6e8a488a None admin] RBAC: Authorizing `identity:get_user(user_id=9effc53793ed40008f1d65d7831cd85e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:57.005326 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb2bab-6315-4c3c-afde-1bdf6e8a488a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:57.007280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 528/1056] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:01:56 2022] GET /identity/v3/users/9effc53793ed40008f1d65d7831cd85e => generated 285 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:57.013010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bf413-564a-4cdb-8185-40413743f215 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:57.125482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bf413-564a-4cdb-8185-40413743f215 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:57.160364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac1645-c5ba-4464-9415-65a0d23ef62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:57.161026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac1645-c5ba-4464-9415-65a0d23ef62e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:57.161257 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac1645-c5ba-4464-9415-65a0d23ef62e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:57.161501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bac1645-c5ba-4464-9415-65a0d23ef62e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:57.184086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:57.234165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac1645-c5ba-4464-9415-65a0d23ef62e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:57.236166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bac1645-c5ba-4464-9415-65a0d23ef62e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:57.341541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 529/1057] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:01:57.343064 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb2bab-6315-4c3c-afde-1bdf6e8a488a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:57.451003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb2bab-6315-4c3c-afde-1bdf6e8a488a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:57.483278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:57.490043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc91d28-a5c3-44f9-8a7a-315c266f6759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:57.490902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc91d28-a5c3-44f9-8a7a-315c266f6759 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:57.491324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc91d28-a5c3-44f9-8a7a-315c266f6759 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:57.491779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc91d28-a5c3-44f9-8a7a-315c266f6759 None admin] PATH_INFO: `/v3/projects/2b3acde8801d476b8a51de3d5f3046b0` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:57.493053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc91d28-a5c3-44f9-8a7a-315c266f6759 None admin] RBAC: Authorizing `identity:get_project(project_id=2b3acde8801d476b8a51de3d5f3046b0)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:57.494616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc91d28-a5c3-44f9-8a7a-315c266f6759 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:57.501136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 529/1058] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:01:57 2022] GET /identity/v3/projects/2b3acde8801d476b8a51de3d5f3046b0 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:57.502704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac1645-c5ba-4464-9415-65a0d23ef62e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:57.558350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:57.609463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bac1645-c5ba-4464-9415-65a0d23ef62e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:57.645352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badc9285-b54f-4f62-ad5a-7dd0f0342f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:57.645951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc9285-b54f-4f62-ad5a-7dd0f0342f8c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:57.646229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc9285-b54f-4f62-ad5a-7dd0f0342f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:57.646501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badc9285-b54f-4f62-ad5a-7dd0f0342f8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:57.647215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc9285-b54f-4f62-ad5a-7dd0f0342f8c None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=2b3acde8801d476b8a51de3d5f3046b0, user.id=9effc53793ed40008f1d65d7831cd85e) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:57.647552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc9285-b54f-4f62-ad5a-7dd0f0342f8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:57.648986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badc9285-b54f-4f62-ad5a-7dd0f0342f8c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:57.663530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 530/1059] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:01:57 2022] GET /identity/v3/role_assignments?user.id=9effc53793ed40008f1d65d7831cd85e&scope.project.id=2b3acde8801d476b8a51de3d5f3046b0&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:58.896416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991104e0-e629-4431-846e-fb23d481a120 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:58.896933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991104e0-e629-4431-846e-fb23d481a120 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:58.896933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991104e0-e629-4431-846e-fb23d481a120 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:58.898349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 530/1060] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:58.903371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56144d53-0a6b-485d-8c7a-ea51c3e2d400 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:58.903797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56144d53-0a6b-485d-8c7a-ea51c3e2d400 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:58.903980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56144d53-0a6b-485d-8c7a-ea51c3e2d400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:58.921888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-56144d53-0a6b-485d-8c7a-ea51c3e2d400 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:58.923073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-56144d53-0a6b-485d-8c7a-ea51c3e2d400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:58.989377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 531/1061] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:59.184143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe1174-0c34-4250-aa1f-e882c3c1c98c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:59.184464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe1174-0c34-4250-aa1f-e882c3c1c98c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:59.184732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe1174-0c34-4250-aa1f-e882c3c1c98c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:59.186166 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 531/1062] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:59.191319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ac62c-0933-44c8-85c0-80e7244d7a14 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:59.191639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ac62c-0933-44c8-85c0-80e7244d7a14 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:59.191920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ac62c-0933-44c8-85c0-80e7244d7a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:59.210428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d99ac62c-0933-44c8-85c0-80e7244d7a14 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:59.211650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d99ac62c-0933-44c8-85c0-80e7244d7a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:01:59.253818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:59.261582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe1174-0c34-4250-aa1f-e882c3c1c98c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:59.281027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 532/1063] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:01:59.370661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe1174-0c34-4250-aa1f-e882c3c1c98c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:59.406339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f078ff-ed67-4d4a-a03e-31e53548dd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:59.406967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f078ff-ed67-4d4a-a03e-31e53548dd9b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:59.407243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f078ff-ed67-4d4a-a03e-31e53548dd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:59.407544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f078ff-ed67-4d4a-a03e-31e53548dd9b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:59.418186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f078ff-ed67-4d4a-a03e-31e53548dd9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-42f078ff-ed67-4d4a-a03e-31e53548dd9b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:01:59.423471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:59.427999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:01:59.428919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 532/1064] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 10 14:01:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:59.438402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ac62c-0933-44c8-85c0-80e7244d7a14 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:59.539853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ac62c-0933-44c8-85c0-80e7244d7a14 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:59.552850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f078ff-ed67-4d4a-a03e-31e53548dd9b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:59.577326 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f369e-5c7e-474a-aee7-7c8fe5ba4acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:59.578198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f369e-5c7e-474a-aee7-7c8fe5ba4acd None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:59.578479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f369e-5c7e-474a-aee7-7c8fe5ba4acd None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:59.579163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f369e-5c7e-474a-aee7-7c8fe5ba4acd None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:59.580269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f369e-5c7e-474a-aee7-7c8fe5ba4acd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:59.580580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f369e-5c7e-474a-aee7-7c8fe5ba4acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:59.582442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16f369e-5c7e-474a-aee7-7c8fe5ba4acd None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:59.597850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 533/1065] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 10 14:01:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:01:59.598895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f369e-5c7e-474a-aee7-7c8fe5ba4acd None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:59.619755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:59.643055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:59.667192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f078ff-ed67-4d4a-a03e-31e53548dd9b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:59.703158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16f369e-5c7e-474a-aee7-7c8fe5ba4acd None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:59.704247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b870ca-d2ab-4a2b-9b36-96b68207940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:59.705005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b870ca-d2ab-4a2b-9b36-96b68207940b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:59.705283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b870ca-d2ab-4a2b-9b36-96b68207940b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:59.705680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b870ca-d2ab-4a2b-9b36-96b68207940b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:59.742512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a4f2b-8104-4469-b518-503f59359a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:59.743144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a4f2b-8104-4469-b518-503f59359a93 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:59.743412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a4f2b-8104-4469-b518-503f59359a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:59.743713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450a4f2b-8104-4469-b518-503f59359a93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:59.754659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a4f2b-8104-4469-b518-503f59359a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:59.756188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450a4f2b-8104-4469-b518-503f59359a93 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-450a4f2b-8104-4469-b518-503f59359a93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:01:59.770528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:01:59.771567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:01:59.772472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:01:59.772472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:01:59.772472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:01:59.772472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 534/1066] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:01:59.777361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a4f2b-8104-4469-b518-503f59359a93 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:59.787901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b870ca-d2ab-4a2b-9b36-96b68207940b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:59.789356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b870ca-d2ab-4a2b-9b36-96b68207940b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:59.852200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:59.873769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:01:59.883776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450a4f2b-8104-4469-b518-503f59359a93 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:01:59.895808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 533/1067] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:01:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:01:59.897163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54184e94-69f4-48e3-8926-9c2e4e3c995b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:59.897465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54184e94-69f4-48e3-8926-9c2e4e3c995b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:59.897739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54184e94-69f4-48e3-8926-9c2e4e3c995b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:59.899105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 534/1068] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:01:59.903819 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54184e94-69f4-48e3-8926-9c2e4e3c995b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:01:59.921069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405509f-fc78-4489-a6d8-eb337946ed73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:01:59.921675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405509f-fc78-4489-a6d8-eb337946ed73 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:59.921954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405509f-fc78-4489-a6d8-eb337946ed73 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:59.922224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405509f-fc78-4489-a6d8-eb337946ed73 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:59.922856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405509f-fc78-4489-a6d8-eb337946ed73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:01:59.923156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405509f-fc78-4489-a6d8-eb337946ed73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:01:59.924585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405509f-fc78-4489-a6d8-eb337946ed73 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:01:59.936191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 535/1069] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:01:59.937360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d3d0-96b9-4e7e-b53e-8a0fce3e2982 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:01:59.937643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d3d0-96b9-4e7e-b53e-8a0fce3e2982 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:01:59.937910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8d3d0-96b9-4e7e-b53e-8a0fce3e2982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:01:59.955683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7df8d3d0-96b9-4e7e-b53e-8a0fce3e2982 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:01:59.957463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7df8d3d0-96b9-4e7e-b53e-8a0fce3e2982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:00.010627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54184e94-69f4-48e3-8926-9c2e4e3c995b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.024959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 536/1070] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:01:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:00.025700 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8d3d0-96b9-4e7e-b53e-8a0fce3e2982 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.050627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd5d6-e755-4631-91ea-b85452e56f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.051265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd5d6-e755-4631-91ea-b85452e56f65 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.051569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd5d6-e755-4631-91ea-b85452e56f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.051841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd5d6-e755-4631-91ea-b85452e56f65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.061996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cd5d6-e755-4631-91ea-b85452e56f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:00.063549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cd5d6-e755-4631-91ea-b85452e56f65 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-872cd5d6-e755-4631-91ea-b85452e56f65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:02:00.078893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:00.079886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:02:00.080818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:02:00.080818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:02:00.080818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:00.080818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 535/1071] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:00.086214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd5d6-e755-4631-91ea-b85452e56f65 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.101337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:00.128359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:00.150485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8d3d0-96b9-4e7e-b53e-8a0fce3e2982 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.191281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d58adb-7b24-4434-9e75-f1a3eff4c856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.191932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd5d6-e755-4631-91ea-b85452e56f65 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.192579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d58adb-7b24-4434-9e75-f1a3eff4c856 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.193060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d58adb-7b24-4434-9e75-f1a3eff4c856 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.193791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d58adb-7b24-4434-9e75-f1a3eff4c856 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.203789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d58adb-7b24-4434-9e75-f1a3eff4c856 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:00.205836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d58adb-7b24-4434-9e75-f1a3eff4c856 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-38d58adb-7b24-4434-9e75-f1a3eff4c856 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:02:00.219825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:00.221057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:02:00.222577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 537/1072] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:00.227730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d58adb-7b24-4434-9e75-f1a3eff4c856 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.229563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45fd3b-5214-4ada-a8e8-45f37824d523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.230177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45fd3b-5214-4ada-a8e8-45f37824d523 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.230459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45fd3b-5214-4ada-a8e8-45f37824d523 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.230733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a45fd3b-5214-4ada-a8e8-45f37824d523 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.231382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45fd3b-5214-4ada-a8e8-45f37824d523 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:00.231715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45fd3b-5214-4ada-a8e8-45f37824d523 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:00.233086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a45fd3b-5214-4ada-a8e8-45f37824d523 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:00.246896 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 536/1073] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:00.300564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45fd3b-5214-4ada-a8e8-45f37824d523 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.345438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d58adb-7b24-4434-9e75-f1a3eff4c856 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.375570 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:00.389221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03245c1-b2c2-4eec-ab59-67cf1b8031f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.389997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03245c1-b2c2-4eec-ab59-67cf1b8031f2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.390281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03245c1-b2c2-4eec-ab59-67cf1b8031f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.390554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03245c1-b2c2-4eec-ab59-67cf1b8031f2 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.391301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03245c1-b2c2-4eec-ab59-67cf1b8031f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:00.391638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03245c1-b2c2-4eec-ab59-67cf1b8031f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:00.394208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03245c1-b2c2-4eec-ab59-67cf1b8031f2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:00.406507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:00.411012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a45fd3b-5214-4ada-a8e8-45f37824d523 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.421200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 538/1074] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:00.422090 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03245c1-b2c2-4eec-ab59-67cf1b8031f2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.458522 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52734d9f-0806-4539-a655-1ab99910bfa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.459397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52734d9f-0806-4539-a655-1ab99910bfa4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.459397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52734d9f-0806-4539-a655-1ab99910bfa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.459617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52734d9f-0806-4539-a655-1ab99910bfa4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.470814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52734d9f-0806-4539-a655-1ab99910bfa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-52734d9f-0806-4539-a655-1ab99910bfa4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:02:00.475926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:00.476890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:00.477732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 537/1075] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:02:00.483489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52734d9f-0806-4539-a655-1ab99910bfa4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.547589 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03245c1-b2c2-4eec-ab59-67cf1b8031f2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.590784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e07cd-f958-4f17-8b6a-2f01fdfee724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.591406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e07cd-f958-4f17-8b6a-2f01fdfee724 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.591711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e07cd-f958-4f17-8b6a-2f01fdfee724 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.591985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908e07cd-f958-4f17-8b6a-2f01fdfee724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.597821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52734d9f-0806-4539-a655-1ab99910bfa4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.602312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e07cd-f958-4f17-8b6a-2f01fdfee724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:00.603771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908e07cd-f958-4f17-8b6a-2f01fdfee724 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-908e07cd-f958-4f17-8b6a-2f01fdfee724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:02:00.618347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:00.619305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:02:00.620634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:02:00.620634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:02:00.620634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:02:00.620634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 539/1076] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:02:00.625176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e07cd-f958-4f17-8b6a-2f01fdfee724 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.626940 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:00.632987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7faff6-b16d-405e-b5d6-bea0adc55af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.633587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7faff6-b16d-405e-b5d6-bea0adc55af0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.633864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7faff6-b16d-405e-b5d6-bea0adc55af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.634137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7faff6-b16d-405e-b5d6-bea0adc55af0 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.634803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7faff6-b16d-405e-b5d6-bea0adc55af0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:00.635107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7faff6-b16d-405e-b5d6-bea0adc55af0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:00.636516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7faff6-b16d-405e-b5d6-bea0adc55af0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:00.650163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 538/1077] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:00.651334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7faff6-b16d-405e-b5d6-bea0adc55af0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.675138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:00.726259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908e07cd-f958-4f17-8b6a-2f01fdfee724 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.751346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7faff6-b16d-405e-b5d6-bea0adc55af0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.759327 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843e5d3-f04d-4fc7-a0c6-48648f365ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.760102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843e5d3-f04d-4fc7-a0c6-48648f365ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.760384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843e5d3-f04d-4fc7-a0c6-48648f365ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.760647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843e5d3-f04d-4fc7-a0c6-48648f365ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.761266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843e5d3-f04d-4fc7-a0c6-48648f365ae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:00.761563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843e5d3-f04d-4fc7-a0c6-48648f365ae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:00.762925 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843e5d3-f04d-4fc7-a0c6-48648f365ae7 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:00.774898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 540/1078] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:00.780931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843e5d3-f04d-4fc7-a0c6-48648f365ae7 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.787413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ba197-ec61-443c-9eb2-73c760b337ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.788071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ba197-ec61-443c-9eb2-73c760b337ed None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.788350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ba197-ec61-443c-9eb2-73c760b337ed None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.788631 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ba197-ec61-443c-9eb2-73c760b337ed None admin] PATH_INFO: `/v3/users/9effc53793ed40008f1d65d7831cd85e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.789762 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ba197-ec61-443c-9eb2-73c760b337ed None admin] RBAC: Authorizing `identity:get_user(user_id=9effc53793ed40008f1d65d7831cd85e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:00.791194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ba197-ec61-443c-9eb2-73c760b337ed None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:00.797515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 539/1079] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/users/9effc53793ed40008f1d65d7831cd85e => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:00.798802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ba197-ec61-443c-9eb2-73c760b337ed None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.860852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:00.879400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843e5d3-f04d-4fc7-a0c6-48648f365ae7 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.891212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:00.901715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ba197-ec61-443c-9eb2-73c760b337ed None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:00.915820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e8ff9-3271-4d1f-8921-65c7f79c52c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.916449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e8ff9-3271-4d1f-8921-65c7f79c52c9 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.916729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e8ff9-3271-4d1f-8921-65c7f79c52c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.917002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1e8ff9-3271-4d1f-8921-65c7f79c52c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.923535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e8ff9-3271-4d1f-8921-65c7f79c52c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:00.925529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1e8ff9-3271-4d1f-8921-65c7f79c52c9 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:00.935590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f05e7d-db82-46ff-83c3-d4eca281af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:00.936206 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f05e7d-db82-46ff-83c3-d4eca281af8c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:00.936481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f05e7d-db82-46ff-83c3-d4eca281af8c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-8d1e8ff9-3271-4d1f-8921-65c7f79c52c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:02:00.936586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:00.937603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:00.938549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:02:00.938549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:02:00.938549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:02:00.938549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:02:00.938549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:02:00.938549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:02:00.938549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:02:00.938549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:02:00.938990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f05e7d-db82-46ff-83c3-d4eca281af8c None admin] PATH_INFO: `/v3/projects/2b3acde8801d476b8a51de3d5f3046b0` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:00.939068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 541/1080] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:02:00.939865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f05e7d-db82-46ff-83c3-d4eca281af8c None admin] RBAC: Authorizing `identity:get_project(project_id=2b3acde8801d476b8a51de3d5f3046b0)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:00.941996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f05e7d-db82-46ff-83c3-d4eca281af8c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:00.948237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 540/1081] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/projects/2b3acde8801d476b8a51de3d5f3046b0 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:00.948387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e8ff9-3271-4d1f-8921-65c7f79c52c9 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:00.949606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f05e7d-db82-46ff-83c3-d4eca281af8c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:01.043743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1e8ff9-3271-4d1f-8921-65c7f79c52c9 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:01.047886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f05e7d-db82-46ff-83c3-d4eca281af8c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:01.077140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea61020-89ca-482f-b7f7-071ab7ee1c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:01.077733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea61020-89ca-482f-b7f7-071ab7ee1c6f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:01.078007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea61020-89ca-482f-b7f7-071ab7ee1c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:01.078278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea61020-89ca-482f-b7f7-071ab7ee1c6f None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:01.078973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea61020-89ca-482f-b7f7-071ab7ee1c6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:01.079245 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea61020-89ca-482f-b7f7-071ab7ee1c6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:01.080708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea61020-89ca-482f-b7f7-071ab7ee1c6f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:01.082546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:01.089159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7955f980-5f57-4d30-9b16-ff5750c14ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:01.089815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955f980-5f57-4d30-9b16-ff5750c14ac0 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:01.090529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955f980-5f57-4d30-9b16-ff5750c14ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:01.090810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7955f980-5f57-4d30-9b16-ff5750c14ac0 None admin] PATH_INFO: `/v3/projects/2b3acde8801d476b8a51de3d5f3046b0/users/9effc53793ed40008f1d65d7831cd85e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:01.092751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955f980-5f57-4d30-9b16-ff5750c14ac0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b3acde8801d476b8a51de3d5f3046b0, user_id=9effc53793ed40008f1d65d7831cd85e, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:01.093059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 542/1082] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:02:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:01.094195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7955f980-5f57-4d30-9b16-ff5750c14ac0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:01.100455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea61020-89ca-482f-b7f7-071ab7ee1c6f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:01.107977 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 541/1083] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:02:00 2022] PUT /identity/v3/projects/2b3acde8801d476b8a51de3d5f3046b0/users/9effc53793ed40008f1d65d7831cd85e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:02:01.137124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:01.211756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea61020-89ca-482f-b7f7-071ab7ee1c6f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:01.247295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d674c-5f05-4293-9fd5-a9e404327b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:01.247998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d674c-5f05-4293-9fd5-a9e404327b02 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:01.248279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d674c-5f05-4293-9fd5-a9e404327b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:01.248554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d674c-5f05-4293-9fd5-a9e404327b02 None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/cb24a1b55ee7408ab24c68991aec54ec/roles/823d0de8de634181af75dabe48e834bb` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:01.250764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d674c-5f05-4293-9fd5-a9e404327b02 None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=cb24a1b55ee7408ab24c68991aec54ec, role_id=823d0de8de634181af75dabe48e834bb)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:01.252262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d674c-5f05-4293-9fd5-a9e404327b02 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:01.266381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 543/1084] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:02:01 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/cb24a1b55ee7408ab24c68991aec54ec/roles/823d0de8de634181af75dabe48e834bb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:02:03.244606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b027f-e411-46bf-9fc9-7344efbd3f52 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:03.245163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b027f-e411-46bf-9fc9-7344efbd3f52 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:03.245163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98b027f-e411-46bf-9fc9-7344efbd3f52 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:03.246458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 542/1085] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:03.251700 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c810c-1969-4bf0-9f27-48169896e72c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:03.252104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c810c-1969-4bf0-9f27-48169896e72c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:03.252266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c810c-1969-4bf0-9f27-48169896e72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:03.271816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-149c810c-1969-4bf0-9f27-48169896e72c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:03.272999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-149c810c-1969-4bf0-9f27-48169896e72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:03.342527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 544/1086] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:03.429845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855aad3e-e7cd-40f2-8a6a-d863d087c9e8 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:03.430231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855aad3e-e7cd-40f2-8a6a-d863d087c9e8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:03.430573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855aad3e-e7cd-40f2-8a6a-d863d087c9e8 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:03.432010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 543/1087] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:03.438317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aebda9-4087-4060-bb66-1dde2d53e1ff None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:03.438999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aebda9-4087-4060-bb66-1dde2d53e1ff None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:03.438999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80aebda9-4087-4060-bb66-1dde2d53e1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:03.458685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-80aebda9-4087-4060-bb66-1dde2d53e1ff None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:03.459898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-80aebda9-4087-4060-bb66-1dde2d53e1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:03.474773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:03.527181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 545/1088] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:02:03.617516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855aad3e-e7cd-40f2-8a6a-d863d087c9e8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:03.717574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855aad3e-e7cd-40f2-8a6a-d863d087c9e8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:03.750613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58624f3e-90ad-4475-afad-f14009c18b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:03.751308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58624f3e-90ad-4475-afad-f14009c18b11 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:03.751682 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58624f3e-90ad-4475-afad-f14009c18b11 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:03.752012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58624f3e-90ad-4475-afad-f14009c18b11 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:03.763132 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58624f3e-90ad-4475-afad-f14009c18b11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-58624f3e-90ad-4475-afad-f14009c18b11 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:02:03.767860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:03.769471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:03.771394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 544/1089] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:02:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:03.775740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aebda9-4087-4060-bb66-1dde2d53e1ff None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:03.793057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58624f3e-90ad-4475-afad-f14009c18b11 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:03.842682 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:03.875854 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80aebda9-4087-4060-bb66-1dde2d53e1ff None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:03.894632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58624f3e-90ad-4475-afad-f14009c18b11 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:03.908972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96513c77-0f8d-4eb1-baea-7c6e11cd73c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:03.909573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96513c77-0f8d-4eb1-baea-7c6e11cd73c6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:03.909842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96513c77-0f8d-4eb1-baea-7c6e11cd73c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:03.910104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96513c77-0f8d-4eb1-baea-7c6e11cd73c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:03.910748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96513c77-0f8d-4eb1-baea-7c6e11cd73c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:03.911039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96513c77-0f8d-4eb1-baea-7c6e11cd73c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:03.912423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96513c77-0f8d-4eb1-baea-7c6e11cd73c6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:03.920185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 546/1090] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:02:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:03.926579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96513c77-0f8d-4eb1-baea-7c6e11cd73c6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:03.928755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d606c6-c7a1-4334-844d-b567277a5497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:03.929387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d606c6-c7a1-4334-844d-b567277a5497 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:03.929669 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d606c6-c7a1-4334-844d-b567277a5497 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:03.929942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d606c6-c7a1-4334-844d-b567277a5497 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:03.941153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d606c6-c7a1-4334-844d-b567277a5497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:02:03.941394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-a1d606c6-c7a1-4334-844d-b567277a5497 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:02:03.945930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:03.946834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:03.947886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 545/1091] 10.222.0.26 () {56 vars in 1120 bytes} [Sat Sep 10 14:02:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:02:03.952869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d606c6-c7a1-4334-844d-b567277a5497 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:04.030484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96513c77-0f8d-4eb1-baea-7c6e11cd73c6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:04.052781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d606c6-c7a1-4334-844d-b567277a5497 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:04.062674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:04.064735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2a558-4ad1-4036-9e61-614ff0787cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:04.065941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2a558-4ad1-4036-9e61-614ff0787cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:04.066432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2a558-4ad1-4036-9e61-614ff0787cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:04.066718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2a558-4ad1-4036-9e61-614ff0787cc0 None admin] PATH_INFO: `/v3/users/9effc53793ed40008f1d65d7831cd85e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:04.067868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2a558-4ad1-4036-9e61-614ff0787cc0 None admin] RBAC: Authorizing `identity:get_user(user_id=9effc53793ed40008f1d65d7831cd85e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:04.069294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df2a558-4ad1-4036-9e61-614ff0787cc0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:04.070863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 547/1092] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:03 2022] GET /identity/v3/users/9effc53793ed40008f1d65d7831cd85e => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:04.076784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2a558-4ad1-4036-9e61-614ff0787cc0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:04.090330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0369f5-aa0e-47a5-bbdd-de2fdbf9eb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:04.090936 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0369f5-aa0e-47a5-bbdd-de2fdbf9eb1b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:04.091216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0369f5-aa0e-47a5-bbdd-de2fdbf9eb1b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:04.091508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0369f5-aa0e-47a5-bbdd-de2fdbf9eb1b None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:04.092177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0369f5-aa0e-47a5-bbdd-de2fdbf9eb1b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:04.092469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0369f5-aa0e-47a5-bbdd-de2fdbf9eb1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:04.093886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0369f5-aa0e-47a5-bbdd-de2fdbf9eb1b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:04.102625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 546/1093] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 10 14:02:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:04.108167 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0369f5-aa0e-47a5-bbdd-de2fdbf9eb1b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:04.161383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:04.180764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df2a558-4ad1-4036-9e61-614ff0787cc0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:04.207469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0369f5-aa0e-47a5-bbdd-de2fdbf9eb1b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:04.214137 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9d3d8-8270-400e-adc0-d10d8c69774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:04.214739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d3d8-8270-400e-adc0-d10d8c69774a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:04.215016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d3d8-8270-400e-adc0-d10d8c69774a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:04.215294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d9d3d8-8270-400e-adc0-d10d8c69774a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:04.240682 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4fb17-9446-4b28-ba86-d5ee0d5571f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:04.241305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4fb17-9446-4b28-ba86-d5ee0d5571f1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:04.241583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4fb17-9446-4b28-ba86-d5ee0d5571f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:04.241857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4fb17-9446-4b28-ba86-d5ee0d5571f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:04.252165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4fb17-9446-4b28-ba86-d5ee0d5571f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:04.254152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c4fb17-9446-4b28-ba86-d5ee0d5571f1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-33c4fb17-9446-4b28-ba86-d5ee0d5571f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:02:04.268163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:04.269741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:02:04.270873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:02:04.270873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:02:04.270873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:04.270873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 547/1094] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:02:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:02:04.275502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4fb17-9446-4b28-ba86-d5ee0d5571f1 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:04.285177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9d3d8-8270-400e-adc0-d10d8c69774a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:04.286629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d9d3d8-8270-400e-adc0-d10d8c69774a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:04.299830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:04.366166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:04.375563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c4fb17-9446-4b28-ba86-d5ee0d5571f1 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:04.386216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 548/1095] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:02:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 10 14:02:04.392158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9d3d8-8270-400e-adc0-d10d8c69774a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:04.409533 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646dde-57e5-4e42-830f-df342fc60093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:04.410113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646dde-57e5-4e42-830f-df342fc60093 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:04.410388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646dde-57e5-4e42-830f-df342fc60093 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:04.410658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40646dde-57e5-4e42-830f-df342fc60093 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:04.411290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646dde-57e5-4e42-830f-df342fc60093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:04.411618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646dde-57e5-4e42-830f-df342fc60093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:04.412988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40646dde-57e5-4e42-830f-df342fc60093 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:04.427965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 548/1096] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:04.433511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646dde-57e5-4e42-830f-df342fc60093 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:04.490615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d9d3d8-8270-400e-adc0-d10d8c69774a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:04.516651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:04.525841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a3388-be32-4c6c-851d-d671f4306a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:04.526460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a3388-be32-4c6c-851d-d671f4306a74 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:04.526740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a3388-be32-4c6c-851d-d671f4306a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:04.527009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a3388-be32-4c6c-851d-d671f4306a74 None admin] PATH_INFO: `/v3/projects/2b3acde8801d476b8a51de3d5f3046b0` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:04.528091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a3388-be32-4c6c-851d-d671f4306a74 None admin] RBAC: Authorizing `identity:get_project(project_id=2b3acde8801d476b8a51de3d5f3046b0)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:04.529527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a3388-be32-4c6c-851d-d671f4306a74 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:04.531228 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 549/1097] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:04 2022] GET /identity/v3/projects/2b3acde8801d476b8a51de3d5f3046b0 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:04.533016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40646dde-57e5-4e42-830f-df342fc60093 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:04.537239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a3388-be32-4c6c-851d-d671f4306a74 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:04.567529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f6832-90c7-4bbf-a7d0-73f865c8b2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:04.568099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6832-90c7-4bbf-a7d0-73f865c8b2ff None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:04.568372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6832-90c7-4bbf-a7d0-73f865c8b2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:04.568638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054f6832-90c7-4bbf-a7d0-73f865c8b2ff None admin] PATH_INFO: `/v3/users/nova` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:04.575573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f6832-90c7-4bbf-a7d0-73f865c8b2ff None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:04.577019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054f6832-90c7-4bbf-a7d0-73f865c8b2ff None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:04.585264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-054f6832-90c7-4bbf-a7d0-73f865c8b2ff None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:02:04.588761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:04.589788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:04.590841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 549/1098] 10.222.0.26 () {56 vars in 1093 bytes} [Sat Sep 10 14:02:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:02:04.595933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f6832-90c7-4bbf-a7d0-73f865c8b2ff None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:04.641965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a3388-be32-4c6c-851d-d671f4306a74 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:04.676589 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c89ed6-08f7-4ce7-bea8-b055164c5d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:04.677350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c89ed6-08f7-4ce7-bea8-b055164c5d8c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:04.677629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c89ed6-08f7-4ce7-bea8-b055164c5d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:04.677898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c89ed6-08f7-4ce7-bea8-b055164c5d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:04.678607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c89ed6-08f7-4ce7-bea8-b055164c5d8c None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=2b3acde8801d476b8a51de3d5f3046b0, user.id=9effc53793ed40008f1d65d7831cd85e) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:04.678910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c89ed6-08f7-4ce7-bea8-b055164c5d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:04.680371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c89ed6-08f7-4ce7-bea8-b055164c5d8c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:04.685944 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 550/1099] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:02:04 2022] GET /identity/v3/role_assignments?user.id=9effc53793ed40008f1d65d7831cd85e&scope.project.id=2b3acde8801d476b8a51de3d5f3046b0&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:04.702251 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054f6832-90c7-4bbf-a7d0-73f865c8b2ff None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:04.740092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92b7c5-2181-4f90-8a70-c4f70fcab3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:04.740720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92b7c5-2181-4f90-8a70-c4f70fcab3b8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:04.740999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92b7c5-2181-4f90-8a70-c4f70fcab3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:04.741272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92b7c5-2181-4f90-8a70-c4f70fcab3b8 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:04.742020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92b7c5-2181-4f90-8a70-c4f70fcab3b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:04.742325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92b7c5-2181-4f90-8a70-c4f70fcab3b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:04.743845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92b7c5-2181-4f90-8a70-c4f70fcab3b8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:04.763597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 550/1100] 10.222.0.26 () {56 vars in 1133 bytes} [Sat Sep 10 14:02:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:04.770366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c89ed6-08f7-4ce7-bea8-b055164c5d8c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:04.875856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c89ed6-08f7-4ce7-bea8-b055164c5d8c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:04.888819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:04.911351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201d1c6-0992-4ab2-abcd-880419028abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:04.911963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201d1c6-0992-4ab2-abcd-880419028abf None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:04.912242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201d1c6-0992-4ab2-abcd-880419028abf None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:04.912513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7201d1c6-0992-4ab2-abcd-880419028abf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:04.978764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201d1c6-0992-4ab2-abcd-880419028abf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:04.980256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7201d1c6-0992-4ab2-abcd-880419028abf None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:05.074771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 551/1101] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:02:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:02:05.080508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92b7c5-2181-4f90-8a70-c4f70fcab3b8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:05.084170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:05.181117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92b7c5-2181-4f90-8a70-c4f70fcab3b8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:05.215026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5eea-1e79-4336-aada-c29a7c516cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:05.215675 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5eea-1e79-4336-aada-c29a7c516cee None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:05.215949 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5eea-1e79-4336-aada-c29a7c516cee None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:05.216214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3a5eea-1e79-4336-aada-c29a7c516cee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:05.226996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a5eea-1e79-4336-aada-c29a7c516cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:05.228478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3a5eea-1e79-4336-aada-c29a7c516cee None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-8f3a5eea-1e79-4336-aada-c29a7c516cee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:02:05.242738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:05.243700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:02:05.244606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:02:05.244606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:02:05.244606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:05.247603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 551/1102] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:02:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:05.249445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201d1c6-0992-4ab2-abcd-880419028abf None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:05.263056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:05.347743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7201d1c6-0992-4ab2-abcd-880419028abf None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:05.379873 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47731657-1cbf-4bf7-ac3a-8650682837dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:05.380468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47731657-1cbf-4bf7-ac3a-8650682837dc None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:05.380743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47731657-1cbf-4bf7-ac3a-8650682837dc None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:05.381009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47731657-1cbf-4bf7-ac3a-8650682837dc None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:05.381638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47731657-1cbf-4bf7-ac3a-8650682837dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:05.381934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47731657-1cbf-4bf7-ac3a-8650682837dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:05.383292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47731657-1cbf-4bf7-ac3a-8650682837dc None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:05.395180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 552/1103] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:05.400840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5eea-1e79-4336-aada-c29a7c516cee None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:05.461590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:05.506738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3a5eea-1e79-4336-aada-c29a7c516cee None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:05.540182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f3030-9e0e-4d6e-bac3-0d2d7ffa2223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:05.540797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f3030-9e0e-4d6e-bac3-0d2d7ffa2223 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:05.541071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f3030-9e0e-4d6e-bac3-0d2d7ffa2223 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:05.541337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01f3030-9e0e-4d6e-bac3-0d2d7ffa2223 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:05.547766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f3030-9e0e-4d6e-bac3-0d2d7ffa2223 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:05.549206 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01f3030-9e0e-4d6e-bac3-0d2d7ffa2223 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c01f3030-9e0e-4d6e-bac3-0d2d7ffa2223 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:02:05.560928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:05.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:05.563432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:02:05.563432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:02:05.563432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:02:05.563432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:02:05.563432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:02:05.563432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:02:05.563432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:02:05.563432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:05.563432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 552/1104] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:02:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:05.568223 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47731657-1cbf-4bf7-ac3a-8650682837dc None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:05.639299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:05.669047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47731657-1cbf-4bf7-ac3a-8650682837dc None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:05.701885 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76affc0-bb21-4495-a534-82750e93b502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:05.702421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76affc0-bb21-4495-a534-82750e93b502 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:05.702693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76affc0-bb21-4495-a534-82750e93b502 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:05.702959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76affc0-bb21-4495-a534-82750e93b502 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:05.703680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76affc0-bb21-4495-a534-82750e93b502 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:05.703977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76affc0-bb21-4495-a534-82750e93b502 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:05.705384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76affc0-bb21-4495-a534-82750e93b502 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:05.716936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 553/1105] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:02:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:05.723807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f3030-9e0e-4d6e-bac3-0d2d7ffa2223 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:05.821161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01f3030-9e0e-4d6e-bac3-0d2d7ffa2223 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:05.836711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:05.858159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e560cd8-f677-423b-a3bd-217e3ad4ad7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:05.858754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560cd8-f677-423b-a3bd-217e3ad4ad7f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:05.859029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560cd8-f677-423b-a3bd-217e3ad4ad7f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:05.859300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e560cd8-f677-423b-a3bd-217e3ad4ad7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:05.860049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e560cd8-f677-423b-a3bd-217e3ad4ad7f None admin] RBAC: Adding query filter params (role.id=823d0de8de634181af75dabe48e834bb, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=cb24a1b55ee7408ab24c68991aec54ec) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:05.860354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e560cd8-f677-423b-a3bd-217e3ad4ad7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:05.861784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e560cd8-f677-423b-a3bd-217e3ad4ad7f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:05.867695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 553/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:02:05 2022] GET /identity/v3/role_assignments?user.id=cb24a1b55ee7408ab24c68991aec54ec&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=823d0de8de634181af75dabe48e834bb => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:06.872647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425549a9-5190-4d6e-bfa4-1fe7cec79d1e None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:06.873170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425549a9-5190-4d6e-bfa4-1fe7cec79d1e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:06.873170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425549a9-5190-4d6e-bfa4-1fe7cec79d1e None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:06.874427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 554/1107] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:06.880479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101268e8-d140-4f90-bf3a-ce8bde6d1e69 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:06.880874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101268e8-d140-4f90-bf3a-ce8bde6d1e69 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:06.881026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101268e8-d140-4f90-bf3a-ce8bde6d1e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:06.899029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-101268e8-d140-4f90-bf3a-ce8bde6d1e69 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:06.900247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-101268e8-d140-4f90-bf3a-ce8bde6d1e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:06.966397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 554/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:07.235270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425549a9-5190-4d6e-bfa4-1fe7cec79d1e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:07.334421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425549a9-5190-4d6e-bfa4-1fe7cec79d1e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:07.363742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:07.368927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d5ac0-efea-4478-acb3-b474230588a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:07.369542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d5ac0-efea-4478-acb3-b474230588a0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:07.369817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d5ac0-efea-4478-acb3-b474230588a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:07.370086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d5ac0-efea-4478-acb3-b474230588a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:07.371166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d5ac0-efea-4478-acb3-b474230588a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:07.372636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669d5ac0-efea-4478-acb3-b474230588a0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:07.374177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 555/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:02:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:07.380479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101268e8-d140-4f90-bf3a-ce8bde6d1e69 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:07.480289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101268e8-d140-4f90-bf3a-ce8bde6d1e69 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:07.489683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:07.514050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49933c4b-7e7c-4cd5-adcf-0516604639ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:07.514643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49933c4b-7e7c-4cd5-adcf-0516604639ae None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:07.514914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49933c4b-7e7c-4cd5-adcf-0516604639ae None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:07.515179 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49933c4b-7e7c-4cd5-adcf-0516604639ae None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:07.515850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49933c4b-7e7c-4cd5-adcf-0516604639ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:07.517259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49933c4b-7e7c-4cd5-adcf-0516604639ae None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:07.537924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 555/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:02:07 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:02:09.713975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8dfe7-44ed-47e8-81bc-11591d350aa4 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:09.714574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8dfe7-44ed-47e8-81bc-11591d350aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:09.714574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d8dfe7-44ed-47e8-81bc-11591d350aa4 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:09.715903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 556/1111] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:09.721521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351f6a1-123f-4c99-bf86-eb82d5f6c9aa None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:09.722162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351f6a1-123f-4c99-bf86-eb82d5f6c9aa None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:09.722162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351f6a1-123f-4c99-bf86-eb82d5f6c9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:09.740284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9351f6a1-123f-4c99-bf86-eb82d5f6c9aa None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:09.741455 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9351f6a1-123f-4c99-bf86-eb82d5f6c9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:09.808649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 556/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:10.075051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8dfe7-44ed-47e8-81bc-11591d350aa4 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:10.174187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d8dfe7-44ed-47e8-81bc-11591d350aa4 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:10.207325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61433c3-e4b0-4d03-b79f-49e44c12c7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:10.207958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61433c3-e4b0-4d03-b79f-49e44c12c7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:10.208234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61433c3-e4b0-4d03-b79f-49e44c12c7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:10.208503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61433c3-e4b0-4d03-b79f-49e44c12c7d4 None admin] PATH_INFO: `/v3/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:10.209875 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61433c3-e4b0-4d03-b79f-49e44c12c7d4 None admin] RBAC: Authorizing `identity:get_role(role_id=9c8bfff8d1a843dbb8aa6482c9fa35bf)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:10.211234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61433c3-e4b0-4d03-b79f-49e44c12c7d4 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:10.212580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 557/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:10 2022] GET /identity/v3/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:10.218431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351f6a1-123f-4c99-bf86-eb82d5f6c9aa None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:10.309214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:10.316923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351f6a1-123f-4c99-bf86-eb82d5f6c9aa None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:10.350085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4b8ba-5b80-4f5a-8a7e-e5ed20e4921d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:10.350637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4b8ba-5b80-4f5a-8a7e-e5ed20e4921d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:10.350909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4b8ba-5b80-4f5a-8a7e-e5ed20e4921d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:10.351174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4b8ba-5b80-4f5a-8a7e-e5ed20e4921d None admin] PATH_INFO: `/v3/users/f0f5e9897f72449ebde7f7d79d96f06f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:10.364079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4b8ba-5b80-4f5a-8a7e-e5ed20e4921d None admin] RBAC: Authorizing `identity:get_user(user_id=f0f5e9897f72449ebde7f7d79d96f06f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:10.365515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4b8ba-5b80-4f5a-8a7e-e5ed20e4921d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:10.367067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 557/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:10 2022] GET /identity/v3/users/f0f5e9897f72449ebde7f7d79d96f06f => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:10.372784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61433c3-e4b0-4d03-b79f-49e44c12c7d4 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:10.436028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:10.468563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61433c3-e4b0-4d03-b79f-49e44c12c7d4 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:10.500988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f0fec-11c6-4e36-9783-72694ea3387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:10.501577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f0fec-11c6-4e36-9783-72694ea3387c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:10.501847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f0fec-11c6-4e36-9783-72694ea3387c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:10.502112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f0fec-11c6-4e36-9783-72694ea3387c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:10.565728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f0fec-11c6-4e36-9783-72694ea3387c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:10.567162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72f0fec-11c6-4e36-9783-72694ea3387c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:10.627351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:10.659745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 558/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:02:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:02:10.665664 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4b8ba-5b80-4f5a-8a7e-e5ed20e4921d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:10.762842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4b8ba-5b80-4f5a-8a7e-e5ed20e4921d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:10.795496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0325313-7c59-414d-9450-8eb35f0d478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:10.796109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0325313-7c59-414d-9450-8eb35f0d478d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:10.796387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0325313-7c59-414d-9450-8eb35f0d478d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:10.796660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0325313-7c59-414d-9450-8eb35f0d478d None admin] PATH_INFO: `/v3/projects/2b3acde8801d476b8a51de3d5f3046b0` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:10.797788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0325313-7c59-414d-9450-8eb35f0d478d None admin] RBAC: Authorizing `identity:get_project(project_id=2b3acde8801d476b8a51de3d5f3046b0)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:10.799223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0325313-7c59-414d-9450-8eb35f0d478d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:10.801380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 558/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:10 2022] GET /identity/v3/projects/2b3acde8801d476b8a51de3d5f3046b0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:10.807422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f0fec-11c6-4e36-9783-72694ea3387c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:10.902803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72f0fec-11c6-4e36-9783-72694ea3387c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:10.935043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d90a7-169f-41c5-9cbb-e6cb643ffb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:10.935653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d90a7-169f-41c5-9cbb-e6cb643ffb02 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:10.935927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d90a7-169f-41c5-9cbb-e6cb643ffb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:10.936195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d90a7-169f-41c5-9cbb-e6cb643ffb02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:10.936895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d90a7-169f-41c5-9cbb-e6cb643ffb02 None admin] RBAC: Adding query filter params (role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf, scope.project.id=2b3acde8801d476b8a51de3d5f3046b0, user.id=f0f5e9897f72449ebde7f7d79d96f06f) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:10.937200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d90a7-169f-41c5-9cbb-e6cb643ffb02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:10.938604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d90a7-169f-41c5-9cbb-e6cb643ffb02 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:10.944459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 559/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:02:10 2022] GET /identity/v3/role_assignments?user.id=f0f5e9897f72449ebde7f7d79d96f06f&scope.project.id=2b3acde8801d476b8a51de3d5f3046b0&role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:13.063773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7897d5d-09b3-4bd0-97c3-4d60dc3862fa None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:13.064320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7897d5d-09b3-4bd0-97c3-4d60dc3862fa None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:13.064320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7897d5d-09b3-4bd0-97c3-4d60dc3862fa None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:13.066161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 559/1118] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:13.071248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02c20-9cd4-4776-943a-85cd7038142c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:13.071685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02c20-9cd4-4776-943a-85cd7038142c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:13.071831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca02c20-9cd4-4776-943a-85cd7038142c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:13.090085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-5ca02c20-9cd4-4776-943a-85cd7038142c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:13.091261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5ca02c20-9cd4-4776-943a-85cd7038142c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:13.116031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:13.156622 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 560/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:13.430336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7897d5d-09b3-4bd0-97c3-4d60dc3862fa None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:13.452745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:13.530984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7897d5d-09b3-4bd0-97c3-4d60dc3862fa None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:13.564051 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd7d87-4318-4116-845c-424aa4dd0699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:13.564668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd7d87-4318-4116-845c-424aa4dd0699 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:13.565548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd7d87-4318-4116-845c-424aa4dd0699 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:13.565824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bd7d87-4318-4116-845c-424aa4dd0699 None admin] PATH_INFO: `/v3/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:13.567214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd7d87-4318-4116-845c-424aa4dd0699 None admin] RBAC: Authorizing `identity:get_role(role_id=9c8bfff8d1a843dbb8aa6482c9fa35bf)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:13.568619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bd7d87-4318-4116-845c-424aa4dd0699 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:13.569922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 560/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:13 2022] GET /identity/v3/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:13.575745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02c20-9cd4-4776-943a-85cd7038142c None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:13.671345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca02c20-9cd4-4776-943a-85cd7038142c None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:13.704478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9814ad2-205f-4960-8db6-3155647c0b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:13.705029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9814ad2-205f-4960-8db6-3155647c0b14 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:13.705306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9814ad2-205f-4960-8db6-3155647c0b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:13.705684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9814ad2-205f-4960-8db6-3155647c0b14 None admin] PATH_INFO: `/v3/users/f0f5e9897f72449ebde7f7d79d96f06f` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:13.706792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9814ad2-205f-4960-8db6-3155647c0b14 None admin] RBAC: Authorizing `identity:get_user(user_id=f0f5e9897f72449ebde7f7d79d96f06f)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:13.708244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9814ad2-205f-4960-8db6-3155647c0b14 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:13.709789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 561/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:13 2022] GET /identity/v3/users/f0f5e9897f72449ebde7f7d79d96f06f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:13.715418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd7d87-4318-4116-845c-424aa4dd0699 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:13.803946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:13.815894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bd7d87-4318-4116-845c-424aa4dd0699 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:13.849578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fd9eb-7076-4e1c-9327-0fdc3ed2a824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:13.850179 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fd9eb-7076-4e1c-9327-0fdc3ed2a824 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:13.850452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fd9eb-7076-4e1c-9327-0fdc3ed2a824 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:13.850722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fd9eb-7076-4e1c-9327-0fdc3ed2a824 None admin] PATH_INFO: `/v3/projects/2b3acde8801d476b8a51de3d5f3046b0` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:13.851843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fd9eb-7076-4e1c-9327-0fdc3ed2a824 None admin] RBAC: Authorizing `identity:get_project(project_id=2b3acde8801d476b8a51de3d5f3046b0)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:13.853271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fd9eb-7076-4e1c-9327-0fdc3ed2a824 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:13.854858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 561/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:13 2022] GET /identity/v3/projects/2b3acde8801d476b8a51de3d5f3046b0 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:13.860788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9814ad2-205f-4960-8db6-3155647c0b14 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:13.891841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:13.958769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9814ad2-205f-4960-8db6-3155647c0b14 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:13.993463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac70ab-1063-4b76-98f9-fbe1d4769162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:13.994098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac70ab-1063-4b76-98f9-fbe1d4769162 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:13.994371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac70ab-1063-4b76-98f9-fbe1d4769162 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:13.994640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac70ab-1063-4b76-98f9-fbe1d4769162 None admin] PATH_INFO: `/v3/projects/2b3acde8801d476b8a51de3d5f3046b0/users/f0f5e9897f72449ebde7f7d79d96f06f/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:13.996746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac70ab-1063-4b76-98f9-fbe1d4769162 None admin] RBAC: Authorizing `identity:create_grant(project_id=2b3acde8801d476b8a51de3d5f3046b0, user_id=f0f5e9897f72449ebde7f7d79d96f06f, role_id=9c8bfff8d1a843dbb8aa6482c9fa35bf)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:13.998157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac70ab-1063-4b76-98f9-fbe1d4769162 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:14.010185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 562/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:02:13 2022] PUT /identity/v3/projects/2b3acde8801d476b8a51de3d5f3046b0/users/f0f5e9897f72449ebde7f7d79d96f06f/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:02:16.125451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff5447-6873-4ea0-b73d-79779e87cb5a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:16.125988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff5447-6873-4ea0-b73d-79779e87cb5a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:16.125988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bff5447-6873-4ea0-b73d-79779e87cb5a None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:16.127291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 562/1124] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:16.133545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c29c3b-b214-483c-adda-111765e3b780 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:16.133992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c29c3b-b214-483c-adda-111765e3b780 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:16.134097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c29c3b-b214-483c-adda-111765e3b780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:16.152536 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c6c29c3b-b214-483c-adda-111765e3b780 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:16.153723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c6c29c3b-b214-483c-adda-111765e3b780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:16.217360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 563/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:16.485269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff5447-6873-4ea0-b73d-79779e87cb5a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:16.583019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bff5447-6873-4ea0-b73d-79779e87cb5a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:16.615586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bb6bf-dc78-4ca5-a784-10387fa82786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:16.616195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bb6bf-dc78-4ca5-a784-10387fa82786 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:16.616468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bb6bf-dc78-4ca5-a784-10387fa82786 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:16.616733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bb6bf-dc78-4ca5-a784-10387fa82786 None admin] PATH_INFO: `/v3/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:16.618081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bb6bf-dc78-4ca5-a784-10387fa82786 None admin] RBAC: Authorizing `identity:get_role(role_id=9c8bfff8d1a843dbb8aa6482c9fa35bf)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:16.619469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531bb6bf-dc78-4ca5-a784-10387fa82786 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:16.620741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 563/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:16 2022] GET /identity/v3/roles/9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:16.626383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c29c3b-b214-483c-adda-111765e3b780 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:16.647343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:16.724431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c29c3b-b214-483c-adda-111765e3b780 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:16.756692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9182fc-66a2-49ae-8b26-5c7abbffc92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:16.757310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9182fc-66a2-49ae-8b26-5c7abbffc92b None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:16.757584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9182fc-66a2-49ae-8b26-5c7abbffc92b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:16.757855 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9182fc-66a2-49ae-8b26-5c7abbffc92b None admin] PATH_INFO: `/v3/users/f0f5e9897f72449ebde7f7d79d96f06f` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:16.758953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9182fc-66a2-49ae-8b26-5c7abbffc92b None admin] RBAC: Authorizing `identity:get_user(user_id=f0f5e9897f72449ebde7f7d79d96f06f)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:16.760410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9182fc-66a2-49ae-8b26-5c7abbffc92b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:16.761964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 564/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:16 2022] GET /identity/v3/users/f0f5e9897f72449ebde7f7d79d96f06f => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:16.767649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bb6bf-dc78-4ca5-a784-10387fa82786 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:16.792574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:16.865654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531bb6bf-dc78-4ca5-a784-10387fa82786 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:16.898157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe709b8-9e64-42ce-bfce-8fe665d3a59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:16.898744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe709b8-9e64-42ce-bfce-8fe665d3a59d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:16.899021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe709b8-9e64-42ce-bfce-8fe665d3a59d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:16.899293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe709b8-9e64-42ce-bfce-8fe665d3a59d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:16.964251 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe709b8-9e64-42ce-bfce-8fe665d3a59d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:16.965696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe709b8-9e64-42ce-bfce-8fe665d3a59d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:16.989987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:17.061725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 564/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:02:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:02:17.068119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9182fc-66a2-49ae-8b26-5c7abbffc92b None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:17.159419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:17.171309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9182fc-66a2-49ae-8b26-5c7abbffc92b None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:17.203586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9863d04-5156-43d3-b86a-4c9e21eb0682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:17.204200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9863d04-5156-43d3-b86a-4c9e21eb0682 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:17.204476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9863d04-5156-43d3-b86a-4c9e21eb0682 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:17.204745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9863d04-5156-43d3-b86a-4c9e21eb0682 None admin] PATH_INFO: `/v3/projects/2b3acde8801d476b8a51de3d5f3046b0` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:17.206466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9863d04-5156-43d3-b86a-4c9e21eb0682 None admin] RBAC: Authorizing `identity:get_project(project_id=2b3acde8801d476b8a51de3d5f3046b0)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:17.207912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9863d04-5156-43d3-b86a-4c9e21eb0682 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:17.209520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 565/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:17 2022] GET /identity/v3/projects/2b3acde8801d476b8a51de3d5f3046b0 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:17.215649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe709b8-9e64-42ce-bfce-8fe665d3a59d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:17.312128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe709b8-9e64-42ce-bfce-8fe665d3a59d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:17.336031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:17.347529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405261c9-835b-450b-b287-f509c3b393a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:17.348121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405261c9-835b-450b-b287-f509c3b393a4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:17.348394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405261c9-835b-450b-b287-f509c3b393a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:17.348663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405261c9-835b-450b-b287-f509c3b393a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:17.349372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405261c9-835b-450b-b287-f509c3b393a4 None admin] RBAC: Adding query filter params (role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf, scope.project.id=2b3acde8801d476b8a51de3d5f3046b0, user.id=f0f5e9897f72449ebde7f7d79d96f06f) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:17.349672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405261c9-835b-450b-b287-f509c3b393a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:17.351096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405261c9-835b-450b-b287-f509c3b393a4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:17.356646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 565/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:02:17 2022] GET /identity/v3/role_assignments?user.id=f0f5e9897f72449ebde7f7d79d96f06f&scope.project.id=2b3acde8801d476b8a51de3d5f3046b0&role.id=9c8bfff8d1a843dbb8aa6482c9fa35bf => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:02:19.530355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3daa6-e715-4cb2-a2dd-eda74aa11116 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:19.530876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3daa6-e715-4cb2-a2dd-eda74aa11116 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:19.530876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3daa6-e715-4cb2-a2dd-eda74aa11116 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:19.532256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 566/1131] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:19.538164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77c2dd-bef1-4f0f-b652-e63c73ceabbe None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:19.538588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77c2dd-bef1-4f0f-b652-e63c73ceabbe None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:19.538697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e77c2dd-bef1-4f0f-b652-e63c73ceabbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:19.556690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3e77c2dd-bef1-4f0f-b652-e63c73ceabbe None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:19.557873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3e77c2dd-bef1-4f0f-b652-e63c73ceabbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:19.622927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 566/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:19.891051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3daa6-e715-4cb2-a2dd-eda74aa11116 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:19.988849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3daa6-e715-4cb2-a2dd-eda74aa11116 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:20.021595 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf94c6e-f2ce-49e5-8aef-5a463881e1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:20.022199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf94c6e-f2ce-49e5-8aef-5a463881e1ea None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:20.022476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf94c6e-f2ce-49e5-8aef-5a463881e1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:20.022745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf94c6e-f2ce-49e5-8aef-5a463881e1ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:20.023868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf94c6e-f2ce-49e5-8aef-5a463881e1ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:20.025290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf94c6e-f2ce-49e5-8aef-5a463881e1ea None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:20.026821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 567/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:02:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:20.032524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77c2dd-bef1-4f0f-b652-e63c73ceabbe None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:20.129898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e77c2dd-bef1-4f0f-b652-e63c73ceabbe None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:20.146527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:20.167672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f11bf9f-6c6a-4d1d-9e34-db2e5bd145b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:20.168270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11bf9f-6c6a-4d1d-9e34-db2e5bd145b6 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:20.168546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11bf9f-6c6a-4d1d-9e34-db2e5bd145b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:20.168814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f11bf9f-6c6a-4d1d-9e34-db2e5bd145b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:20.169450 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11bf9f-6c6a-4d1d-9e34-db2e5bd145b6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:20.170877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f11bf9f-6c6a-4d1d-9e34-db2e5bd145b6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:20.197685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 567/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:02:20 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:02:22.349191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c5360-382b-4bb7-8688-56ce29fcd5a4 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:22.349722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c5360-382b-4bb7-8688-56ce29fcd5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:22.349722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5c5360-382b-4bb7-8688-56ce29fcd5a4 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:22.351061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 568/1135] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:22.356484 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dae67-494b-4bb9-99f9-3111154917ce None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:22.356925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dae67-494b-4bb9-99f9-3111154917ce None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:22.357020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506dae67-494b-4bb9-99f9-3111154917ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:22.375154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-506dae67-494b-4bb9-99f9-3111154917ce None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:22.376338 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-506dae67-494b-4bb9-99f9-3111154917ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:22.441421 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 568/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:22.711615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c5360-382b-4bb7-8688-56ce29fcd5a4 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:22.736736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:22.810333 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5c5360-382b-4bb7-8688-56ce29fcd5a4 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:22.844262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73428788-bb8f-40d3-9035-f379a6aaefa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:22.844880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73428788-bb8f-40d3-9035-f379a6aaefa5 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:22.845158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73428788-bb8f-40d3-9035-f379a6aaefa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:22.845427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73428788-bb8f-40d3-9035-f379a6aaefa5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:22.846820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73428788-bb8f-40d3-9035-f379a6aaefa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:22.848309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73428788-bb8f-40d3-9035-f379a6aaefa5 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:22.849874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 569/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:02:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:22.855607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506dae67-494b-4bb9-99f9-3111154917ce None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:22.950466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:22.953313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506dae67-494b-4bb9-99f9-3111154917ce None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:22.988682 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f097620-38db-40d7-a549-dae2ee62cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:22.989201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f097620-38db-40d7-a549-dae2ee62cac4 None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:22.989472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f097620-38db-40d7-a549-dae2ee62cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:22.989737 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f097620-38db-40d7-a549-dae2ee62cac4 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:22.990367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f097620-38db-40d7-a549-dae2ee62cac4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:22.991821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f097620-38db-40d7-a549-dae2ee62cac4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:23.012105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 569/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:02:22 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:02:25.204861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d1a2e-959b-4d05-b8d3-36a3528e96d7 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:25.206045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d1a2e-959b-4d05-b8d3-36a3528e96d7 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:25.206045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950d1a2e-959b-4d05-b8d3-36a3528e96d7 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:25.207669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 570/1139] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:25.213065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a2dcc-466c-4e30-a01e-0fa5ab5ac77c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:25.213495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a2dcc-466c-4e30-a01e-0fa5ab5ac77c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:25.213600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a2dcc-466c-4e30-a01e-0fa5ab5ac77c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:25.231741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fb5a2dcc-466c-4e30-a01e-0fa5ab5ac77c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:25.232915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fb5a2dcc-466c-4e30-a01e-0fa5ab5ac77c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:25.297189 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 570/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:25.568370 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d1a2e-959b-4d05-b8d3-36a3528e96d7 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:25.657792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:25.667748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950d1a2e-959b-4d05-b8d3-36a3528e96d7 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:25.700791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33256106-3b48-460f-873c-038bc67120b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:25.701404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33256106-3b48-460f-873c-038bc67120b3 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:25.701682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33256106-3b48-460f-873c-038bc67120b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:25.701953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33256106-3b48-460f-873c-038bc67120b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:25.711712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33256106-3b48-460f-873c-038bc67120b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-33256106-3b48-460f-873c-038bc67120b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:02:25.716483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:25.717650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:02:25.718747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 571/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:02:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:25.724283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a2dcc-466c-4e30-a01e-0fa5ab5ac77c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:25.811468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:25.827517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a2dcc-466c-4e30-a01e-0fa5ab5ac77c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:25.860785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716938f4-7ae4-4d37-a6f1-26f509769224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:25.861385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716938f4-7ae4-4d37-a6f1-26f509769224 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:25.861853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716938f4-7ae4-4d37-a6f1-26f509769224 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:25.862128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716938f4-7ae4-4d37-a6f1-26f509769224 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:25.862790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716938f4-7ae4-4d37-a6f1-26f509769224 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:25.863091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716938f4-7ae4-4d37-a6f1-26f509769224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:25.864492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716938f4-7ae4-4d37-a6f1-26f509769224 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:25.872426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 571/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:02:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:25.879086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33256106-3b48-460f-873c-038bc67120b3 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:25.975022 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33256106-3b48-460f-873c-038bc67120b3 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:26.007728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769fa4d-39e5-40ca-a869-e56b10976779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:26.008347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769fa4d-39e5-40ca-a869-e56b10976779 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:26.008626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769fa4d-39e5-40ca-a869-e56b10976779 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:26.008897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769fa4d-39e5-40ca-a869-e56b10976779 None admin] PATH_INFO: `/v3/users/f7595d53172441f6aaa4e02ed4432deb` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:26.022893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769fa4d-39e5-40ca-a869-e56b10976779 None admin] RBAC: Authorizing `identity:get_user(user_id=f7595d53172441f6aaa4e02ed4432deb)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:26.024353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769fa4d-39e5-40ca-a869-e56b10976779 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:26.025933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 572/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:25 2022] GET /identity/v3/users/f7595d53172441f6aaa4e02ed4432deb => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:26.031687 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716938f4-7ae4-4d37-a6f1-26f509769224 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:26.129460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716938f4-7ae4-4d37-a6f1-26f509769224 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:26.162811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31dffe-a73f-4676-a9d1-2ede189f9351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:26.163403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31dffe-a73f-4676-a9d1-2ede189f9351 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:26.163715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31dffe-a73f-4676-a9d1-2ede189f9351 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:26.163986 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31dffe-a73f-4676-a9d1-2ede189f9351 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:26.182502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:26.232312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31dffe-a73f-4676-a9d1-2ede189f9351 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:26.233767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31dffe-a73f-4676-a9d1-2ede189f9351 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:26.329335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 572/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:02:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:02:26.335291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769fa4d-39e5-40ca-a869-e56b10976779 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:26.347552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:26.432464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769fa4d-39e5-40ca-a869-e56b10976779 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:26.465449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8022c13b-012a-46b5-8889-3b20cebb26ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:26.466069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022c13b-012a-46b5-8889-3b20cebb26ce None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:26.466343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022c13b-012a-46b5-8889-3b20cebb26ce None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:26.466616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8022c13b-012a-46b5-8889-3b20cebb26ce None admin] PATH_INFO: `/v3/projects/4fe905c6601a4405a2e0e1d89ba3c021` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:26.467733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022c13b-012a-46b5-8889-3b20cebb26ce None admin] RBAC: Authorizing `identity:get_project(project_id=4fe905c6601a4405a2e0e1d89ba3c021)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:26.469149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8022c13b-012a-46b5-8889-3b20cebb26ce None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:26.470784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 573/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:26 2022] GET /identity/v3/projects/4fe905c6601a4405a2e0e1d89ba3c021 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:26.477715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31dffe-a73f-4676-a9d1-2ede189f9351 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:26.524821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:26.574614 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31dffe-a73f-4676-a9d1-2ede189f9351 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:26.607570 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab9e79-f4f1-49ca-a530-ad87999fed8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:26.608161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9e79-f4f1-49ca-a530-ad87999fed8c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:26.608438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9e79-f4f1-49ca-a530-ad87999fed8c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:26.608712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab9e79-f4f1-49ca-a530-ad87999fed8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:26.609426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9e79-f4f1-49ca-a530-ad87999fed8c None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=4fe905c6601a4405a2e0e1d89ba3c021, user.id=f7595d53172441f6aaa4e02ed4432deb) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:26.609733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9e79-f4f1-49ca-a530-ad87999fed8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:26.611162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab9e79-f4f1-49ca-a530-ad87999fed8c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:26.616909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 573/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:02:26 2022] GET /identity/v3/role_assignments?user.id=f7595d53172441f6aaa4e02ed4432deb&scope.project.id=4fe905c6601a4405a2e0e1d89ba3c021&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:28.728801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4930a9-254d-41b4-938a-97786d75b463 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:28.729347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4930a9-254d-41b4-938a-97786d75b463 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:28.729347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4930a9-254d-41b4-938a-97786d75b463 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:28.730626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 574/1147] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:28.735778 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc880c-fcf8-4615-9d94-433a4c014a3f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:28.736209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc880c-fcf8-4615-9d94-433a4c014a3f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:28.736308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc880c-fcf8-4615-9d94-433a4c014a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:28.754989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2ffc880c-fcf8-4615-9d94-433a4c014a3f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:28.756198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2ffc880c-fcf8-4615-9d94-433a4c014a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:28.820557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 574/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:29.092941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4930a9-254d-41b4-938a-97786d75b463 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:29.170335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:29.192072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4930a9-254d-41b4-938a-97786d75b463 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:29.225561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894ebe-6040-454d-a06c-b9e11893f635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:29.226161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894ebe-6040-454d-a06c-b9e11893f635 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:29.226436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894ebe-6040-454d-a06c-b9e11893f635 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:29.226699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06894ebe-6040-454d-a06c-b9e11893f635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:29.236231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06894ebe-6040-454d-a06c-b9e11893f635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-06894ebe-6040-454d-a06c-b9e11893f635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:02:29.241029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:29.242211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:02:29.243119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 575/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:02:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:29.248187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc880c-fcf8-4615-9d94-433a4c014a3f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:29.290901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:29.345158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc880c-fcf8-4615-9d94-433a4c014a3f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:29.379219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce85eb4-147e-49c0-837f-04bcd0807b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:29.379835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85eb4-147e-49c0-837f-04bcd0807b9d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:29.380108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85eb4-147e-49c0-837f-04bcd0807b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:29.380374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce85eb4-147e-49c0-837f-04bcd0807b9d None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:29.381020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce85eb4-147e-49c0-837f-04bcd0807b9d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:29.381315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce85eb4-147e-49c0-837f-04bcd0807b9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:29.382703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce85eb4-147e-49c0-837f-04bcd0807b9d None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:29.390366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 575/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:02:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:29.396317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894ebe-6040-454d-a06c-b9e11893f635 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:29.494556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06894ebe-6040-454d-a06c-b9e11893f635 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:29.527065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a19e8-2328-46f5-9ea3-e20fc3d589a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:29.527712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a19e8-2328-46f5-9ea3-e20fc3d589a2 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:29.527990 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a19e8-2328-46f5-9ea3-e20fc3d589a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:29.528261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969a19e8-2328-46f5-9ea3-e20fc3d589a2 None admin] PATH_INFO: `/v3/users/f7595d53172441f6aaa4e02ed4432deb` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:29.529362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a19e8-2328-46f5-9ea3-e20fc3d589a2 None admin] RBAC: Authorizing `identity:get_user(user_id=f7595d53172441f6aaa4e02ed4432deb)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:29.530791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969a19e8-2328-46f5-9ea3-e20fc3d589a2 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:29.532399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 576/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:29 2022] GET /identity/v3/users/f7595d53172441f6aaa4e02ed4432deb => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:29.539879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce85eb4-147e-49c0-837f-04bcd0807b9d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:29.636619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce85eb4-147e-49c0-837f-04bcd0807b9d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:29.638964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:29.670268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d3d537-c0fd-4ffa-b68f-ed30b7fb3ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:29.670885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3d537-c0fd-4ffa-b68f-ed30b7fb3ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:29.671162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3d537-c0fd-4ffa-b68f-ed30b7fb3ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:29.671434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d3d537-c0fd-4ffa-b68f-ed30b7fb3ae6 None admin] PATH_INFO: `/v3/projects/4fe905c6601a4405a2e0e1d89ba3c021` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:29.672571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3d537-c0fd-4ffa-b68f-ed30b7fb3ae6 None admin] RBAC: Authorizing `identity:get_project(project_id=4fe905c6601a4405a2e0e1d89ba3c021)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:29.674556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d3d537-c0fd-4ffa-b68f-ed30b7fb3ae6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:29.676353 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 576/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:29 2022] GET /identity/v3/projects/4fe905c6601a4405a2e0e1d89ba3c021 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:29.682394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a19e8-2328-46f5-9ea3-e20fc3d589a2 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:29.691263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:29.778647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969a19e8-2328-46f5-9ea3-e20fc3d589a2 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:29.810576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5b663-28e1-498d-9d08-e3453a2e0d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:29.811203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5b663-28e1-498d-9d08-e3453a2e0d31 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:29.811501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5b663-28e1-498d-9d08-e3453a2e0d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:29.811772 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5b663-28e1-498d-9d08-e3453a2e0d31 None admin] PATH_INFO: `/v3/projects/4fe905c6601a4405a2e0e1d89ba3c021/users/f7595d53172441f6aaa4e02ed4432deb/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:29.813772 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5b663-28e1-498d-9d08-e3453a2e0d31 None admin] RBAC: Authorizing `identity:create_grant(project_id=4fe905c6601a4405a2e0e1d89ba3c021, user_id=f7595d53172441f6aaa4e02ed4432deb, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:29.815801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5b663-28e1-498d-9d08-e3453a2e0d31 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:29.828618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 577/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:02:29 2022] PUT /identity/v3/projects/4fe905c6601a4405a2e0e1d89ba3c021/users/f7595d53172441f6aaa4e02ed4432deb/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:02:31.977987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b3f82-1291-445b-8fde-9a50bb472b30 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:31.978535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b3f82-1291-445b-8fde-9a50bb472b30 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:31.978535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b3f82-1291-445b-8fde-9a50bb472b30 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:31.979875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 577/1154] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:31.985276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41da68-fe32-4f9b-972c-0f8e5baca8a5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:31.985653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41da68-fe32-4f9b-972c-0f8e5baca8a5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:31.985824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a41da68-fe32-4f9b-972c-0f8e5baca8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:32.004844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2a41da68-fe32-4f9b-972c-0f8e5baca8a5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:32.006018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2a41da68-fe32-4f9b-972c-0f8e5baca8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:32.063097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:32.075472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 578/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:32.349647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b3f82-1291-445b-8fde-9a50bb472b30 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:32.455158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b3f82-1291-445b-8fde-9a50bb472b30 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:32.488836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57623aa2-25d6-4b96-b667-d3b399b356ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:32.489457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57623aa2-25d6-4b96-b667-d3b399b356ec None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:32.489733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57623aa2-25d6-4b96-b667-d3b399b356ec None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:32.490003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57623aa2-25d6-4b96-b667-d3b399b356ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:32.500540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57623aa2-25d6-4b96-b667-d3b399b356ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-57623aa2-25d6-4b96-b667-d3b399b356ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:02:32.505380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:32.506526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:32.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 578/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:02:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:32.512595 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41da68-fe32-4f9b-972c-0f8e5baca8a5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:32.619335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a41da68-fe32-4f9b-972c-0f8e5baca8a5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:32.652942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6403f3f-183e-401a-8841-7a65036ffb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:32.653558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6403f3f-183e-401a-8841-7a65036ffb84 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:32.653836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6403f3f-183e-401a-8841-7a65036ffb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:32.654103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6403f3f-183e-401a-8841-7a65036ffb84 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:32.654769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6403f3f-183e-401a-8841-7a65036ffb84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:32.655070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6403f3f-183e-401a-8841-7a65036ffb84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:32.656482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6403f3f-183e-401a-8841-7a65036ffb84 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:32.665414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 579/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:02:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:32.671693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57623aa2-25d6-4b96-b667-d3b399b356ec None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:32.706533 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:32.779736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57623aa2-25d6-4b96-b667-d3b399b356ec None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:32.816553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e372aa-8f8f-445c-afec-0ff431f7fa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:32.817186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e372aa-8f8f-445c-afec-0ff431f7fa0e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:32.817465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e372aa-8f8f-445c-afec-0ff431f7fa0e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:32.817777 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e372aa-8f8f-445c-afec-0ff431f7fa0e None admin] PATH_INFO: `/v3/users/f7595d53172441f6aaa4e02ed4432deb` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:32.818947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e372aa-8f8f-445c-afec-0ff431f7fa0e None admin] RBAC: Authorizing `identity:get_user(user_id=f7595d53172441f6aaa4e02ed4432deb)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:32.820437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e372aa-8f8f-445c-afec-0ff431f7fa0e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:32.822024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 579/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:32 2022] GET /identity/v3/users/f7595d53172441f6aaa4e02ed4432deb => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:32.828428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6403f3f-183e-401a-8841-7a65036ffb84 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:32.891692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:32.942544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6403f3f-183e-401a-8841-7a65036ffb84 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:32.982177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ef04a-2631-40c8-9fc8-f5948aee7972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:32.982785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ef04a-2631-40c8-9fc8-f5948aee7972 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:32.983106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ef04a-2631-40c8-9fc8-f5948aee7972 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:32.983375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8ef04a-2631-40c8-9fc8-f5948aee7972 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:33.059530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ef04a-2631-40c8-9fc8-f5948aee7972 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:33.061137 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8ef04a-2631-40c8-9fc8-f5948aee7972 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:33.119465 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:33.167596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 580/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:02:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:02:33.174334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e372aa-8f8f-445c-afec-0ff431f7fa0e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:33.270232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:33.273515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e372aa-8f8f-445c-afec-0ff431f7fa0e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:33.306271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8494d6-2f19-49fa-8113-f8bfb8177e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:33.306882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8494d6-2f19-49fa-8113-f8bfb8177e98 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:33.307155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8494d6-2f19-49fa-8113-f8bfb8177e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:33.307422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8494d6-2f19-49fa-8113-f8bfb8177e98 None admin] PATH_INFO: `/v3/projects/4fe905c6601a4405a2e0e1d89ba3c021` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:33.308540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8494d6-2f19-49fa-8113-f8bfb8177e98 None admin] RBAC: Authorizing `identity:get_project(project_id=4fe905c6601a4405a2e0e1d89ba3c021)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:33.309963 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8494d6-2f19-49fa-8113-f8bfb8177e98 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:33.311584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 580/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:33 2022] GET /identity/v3/projects/4fe905c6601a4405a2e0e1d89ba3c021 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:33.318548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ef04a-2631-40c8-9fc8-f5948aee7972 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:33.414336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8ef04a-2631-40c8-9fc8-f5948aee7972 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:33.447887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ec09b-b42a-4bd3-bfe8-4117cf9d0d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:33.448492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ec09b-b42a-4bd3-bfe8-4117cf9d0d3c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:33.448766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ec09b-b42a-4bd3-bfe8-4117cf9d0d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:33.449034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ec09b-b42a-4bd3-bfe8-4117cf9d0d3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:33.449746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ec09b-b42a-4bd3-bfe8-4117cf9d0d3c None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=4fe905c6601a4405a2e0e1d89ba3c021, user.id=f7595d53172441f6aaa4e02ed4432deb) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:33.450049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ec09b-b42a-4bd3-bfe8-4117cf9d0d3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:33.451480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ec09b-b42a-4bd3-bfe8-4117cf9d0d3c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:33.457672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 581/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:02:33 2022] GET /identity/v3/role_assignments?user.id=f7595d53172441f6aaa4e02ed4432deb&scope.project.id=4fe905c6601a4405a2e0e1d89ba3c021&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:35.616654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c068f1c-242a-440d-95f8-0179c44858ed None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:35.617199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c068f1c-242a-440d-95f8-0179c44858ed None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:35.617199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c068f1c-242a-440d-95f8-0179c44858ed None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:35.618505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 581/1162] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:35.623803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776738-675a-43b5-a11d-0cea6195cca0 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:35.624222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776738-675a-43b5-a11d-0cea6195cca0 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:35.624351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776738-675a-43b5-a11d-0cea6195cca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:35.642085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-4f776738-675a-43b5-a11d-0cea6195cca0 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:35.643264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4f776738-675a-43b5-a11d-0cea6195cca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:35.659714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:35.708167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 582/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:35.975902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c068f1c-242a-440d-95f8-0179c44858ed None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:36.074370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c068f1c-242a-440d-95f8-0179c44858ed None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:36.105954 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b098ad-f87e-42bc-b5c8-16b82a2f9b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:36.106534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b098ad-f87e-42bc-b5c8-16b82a2f9b24 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:36.106804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b098ad-f87e-42bc-b5c8-16b82a2f9b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:36.107067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b098ad-f87e-42bc-b5c8-16b82a2f9b24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:36.140031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:36.171096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b098ad-f87e-42bc-b5c8-16b82a2f9b24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:36.172561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b098ad-f87e-42bc-b5c8-16b82a2f9b24 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:36.267111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 582/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:02:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:02:36.273116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f776738-675a-43b5-a11d-0cea6195cca0 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:36.368121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f776738-675a-43b5-a11d-0cea6195cca0 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:36.399862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d33461b-4c98-4f70-b6a6-6aff2738a7ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:36.400477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33461b-4c98-4f70-b6a6-6aff2738a7ed None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:36.400754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33461b-4c98-4f70-b6a6-6aff2738a7ed None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:36.401021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d33461b-4c98-4f70-b6a6-6aff2738a7ed None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:36.407373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33461b-4c98-4f70-b6a6-6aff2738a7ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:36.409364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d33461b-4c98-4f70-b6a6-6aff2738a7ed None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-2d33461b-4c98-4f70-b6a6-6aff2738a7ed None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:36.420372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:36.421561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:02:36.422563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 583/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 10 14:02:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:36.427512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b098ad-f87e-42bc-b5c8-16b82a2f9b24 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:36.492143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:36.525522 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b098ad-f87e-42bc-b5c8-16b82a2f9b24 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:36.558803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e89cf-04af-43bc-b877-209f146793c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:36.559385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e89cf-04af-43bc-b877-209f146793c7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:36.559682 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e89cf-04af-43bc-b877-209f146793c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:36.559947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e89cf-04af-43bc-b877-209f146793c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:36.560569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e89cf-04af-43bc-b877-209f146793c7 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:36.560861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e89cf-04af-43bc-b877-209f146793c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:36.562236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e89cf-04af-43bc-b877-209f146793c7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:36.569868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 583/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Sat Sep 10 14:02:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:38.694208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829951a4-1bb0-499d-85bb-6651dd99fd85 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:38.694746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829951a4-1bb0-499d-85bb-6651dd99fd85 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:38.694746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829951a4-1bb0-499d-85bb-6651dd99fd85 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:38.696045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 584/1167] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:38.701164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fcc65-b43e-4317-b921-ef171737d160 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:38.701575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fcc65-b43e-4317-b921-ef171737d160 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:38.701709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fcc65-b43e-4317-b921-ef171737d160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:38.720257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-654fcc65-b43e-4317-b921-ef171737d160 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:38.721915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-654fcc65-b43e-4317-b921-ef171737d160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:38.786588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 584/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:39.054457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829951a4-1bb0-499d-85bb-6651dd99fd85 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:39.069667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:39.153896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829951a4-1bb0-499d-85bb-6651dd99fd85 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:39.186702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c489e4-451a-4d8b-b2c2-e22d8b947b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:39.187291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c489e4-451a-4d8b-b2c2-e22d8b947b51 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:39.187599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c489e4-451a-4d8b-b2c2-e22d8b947b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:39.187873 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c489e4-451a-4d8b-b2c2-e22d8b947b51 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:39.188539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c489e4-451a-4d8b-b2c2-e22d8b947b51 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:39.190199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c489e4-451a-4d8b-b2c2-e22d8b947b51 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:39.207343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 585/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Sat Sep 10 14:02:39 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:02:41.366531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d70e7a-5476-4e26-a13e-7c06bf3c9473 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:41.367069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d70e7a-5476-4e26-a13e-7c06bf3c9473 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:41.367069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d70e7a-5476-4e26-a13e-7c06bf3c9473 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:41.368454 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 585/1170] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:41.373646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee3261-d074-4da0-bfdb-72f25111d78e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:41.374075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee3261-d074-4da0-bfdb-72f25111d78e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:41.374190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cee3261-d074-4da0-bfdb-72f25111d78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:41.392124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-0cee3261-d074-4da0-bfdb-72f25111d78e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:41.393926 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0cee3261-d074-4da0-bfdb-72f25111d78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:41.461179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 586/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:41.728940 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d70e7a-5476-4e26-a13e-7c06bf3c9473 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:41.789278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:41.827404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d70e7a-5476-4e26-a13e-7c06bf3c9473 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:41.860707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910f2bde-9601-4d79-a408-28a9c19c3a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:41.861899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f2bde-9601-4d79-a408-28a9c19c3a01 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:41.862344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f2bde-9601-4d79-a408-28a9c19c3a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:41.862620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910f2bde-9601-4d79-a408-28a9c19c3a01 None admin] PATH_INFO: `/v3/domains/da834d0c5cdf496b9bac5a5d531b484b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:41.872577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f2bde-9601-4d79-a408-28a9c19c3a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=da834d0c5cdf496b9bac5a5d531b484b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:41.874012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910f2bde-9601-4d79-a408-28a9c19c3a01 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:41.875571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 586/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Sat Sep 10 14:02:41 2022] GET /identity/v3/domains/da834d0c5cdf496b9bac5a5d531b484b => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:41.882004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee3261-d074-4da0-bfdb-72f25111d78e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:41.884182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:41.978340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cee3261-d074-4da0-bfdb-72f25111d78e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:42.011790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e5a61-6675-456c-b83d-dba7dc778786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:42.012321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e5a61-6675-456c-b83d-dba7dc778786 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:42.012597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e5a61-6675-456c-b83d-dba7dc778786 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:42.012864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e5a61-6675-456c-b83d-dba7dc778786 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:42.013506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e5a61-6675-456c-b83d-dba7dc778786 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:42.014919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e5a61-6675-456c-b83d-dba7dc778786 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:42.043118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 587/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:02:41 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:02:44.198715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2110d6-29a3-47bb-aca8-a958b04ac005 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:44.199241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2110d6-29a3-47bb-aca8-a958b04ac005 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:44.199241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2110d6-29a3-47bb-aca8-a958b04ac005 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:44.200730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 587/1174] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:44.206379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3cac1-f3bc-48f6-9ebe-e7ae17360186 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:44.206790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3cac1-f3bc-48f6-9ebe-e7ae17360186 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:44.206915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3cac1-f3bc-48f6-9ebe-e7ae17360186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:44.225040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a7c3cac1-f3bc-48f6-9ebe-e7ae17360186 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:44.226199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a7c3cac1-f3bc-48f6-9ebe-e7ae17360186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:44.274614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:44.292998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 588/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:44.563901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2110d6-29a3-47bb-aca8-a958b04ac005 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:44.663700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2110d6-29a3-47bb-aca8-a958b04ac005 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:44.685140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:44.697442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f2f93b-251e-41fe-8e36-ea988ed42666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:44.698056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2f93b-251e-41fe-8e36-ea988ed42666 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:44.698332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2f93b-251e-41fe-8e36-ea988ed42666 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:44.698600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f2f93b-251e-41fe-8e36-ea988ed42666 None admin] PATH_INFO: `/v3/domains/da834d0c5cdf496b9bac5a5d531b484b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:44.699722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2f93b-251e-41fe-8e36-ea988ed42666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=da834d0c5cdf496b9bac5a5d531b484b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:44.701158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f2f93b-251e-41fe-8e36-ea988ed42666 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:44.702684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 588/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Sat Sep 10 14:02:44 2022] GET /identity/v3/domains/da834d0c5cdf496b9bac5a5d531b484b => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:44.709058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3cac1-f3bc-48f6-9ebe-e7ae17360186 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:44.803151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3cac1-f3bc-48f6-9ebe-e7ae17360186 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:44.836258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723b050-3d39-42a4-938f-c8b41a5422e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:44.836780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723b050-3d39-42a4-938f-c8b41a5422e9 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:44.837053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723b050-3d39-42a4-938f-c8b41a5422e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:44.837319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723b050-3d39-42a4-938f-c8b41a5422e9 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:44.837952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723b050-3d39-42a4-938f-c8b41a5422e9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:44.839363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a723b050-3d39-42a4-938f-c8b41a5422e9 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:44.860152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 589/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:02:44 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:02:47.016038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d205873-41ae-46c3-867b-f21ca60a4ebe None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:47.016576 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d205873-41ae-46c3-867b-f21ca60a4ebe None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:47.016576 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d205873-41ae-46c3-867b-f21ca60a4ebe None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:47.018492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 589/1178] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:47.023854 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc2f1f-e0b7-4e99-90c7-5e670455a580 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:47.024262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc2f1f-e0b7-4e99-90c7-5e670455a580 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:47.024369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc2f1f-e0b7-4e99-90c7-5e670455a580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:47.042236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-4adc2f1f-e0b7-4e99-90c7-5e670455a580 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:47.043404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4adc2f1f-e0b7-4e99-90c7-5e670455a580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:47.075917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:47.108764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 590/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:47.382271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d205873-41ae-46c3-867b-f21ca60a4ebe None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:47.481653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d205873-41ae-46c3-867b-f21ca60a4ebe None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:47.514147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154db729-878e-4adc-8775-1012f1649861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:47.514757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154db729-878e-4adc-8775-1012f1649861 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:47.515026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154db729-878e-4adc-8775-1012f1649861 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:47.515291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154db729-878e-4adc-8775-1012f1649861 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:47.526088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154db729-878e-4adc-8775-1012f1649861 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-154db729-878e-4adc-8775-1012f1649861 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:02:47.530785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:47.531901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:47.533097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 590/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:02:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:47.537973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc2f1f-e0b7-4e99-90c7-5e670455a580 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:47.632918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc2f1f-e0b7-4e99-90c7-5e670455a580 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:47.667319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c097-0e03-4269-8e18-2e55165c2f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:47.667319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c097-0e03-4269-8e18-2e55165c2f03 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:47.667606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c097-0e03-4269-8e18-2e55165c2f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:47.667728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6629c097-0e03-4269-8e18-2e55165c2f03 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:47.669523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629c097-0e03-4269-8e18-2e55165c2f03 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:47.669523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629c097-0e03-4269-8e18-2e55165c2f03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:47.670046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6629c097-0e03-4269-8e18-2e55165c2f03 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:47.679289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 591/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:02:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:47.685306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154db729-878e-4adc-8775-1012f1649861 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:47.733850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:47.783822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154db729-878e-4adc-8775-1012f1649861 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:47.817609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d2c5a-a14b-41a0-b208-c964c45cdc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:47.818224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d2c5a-a14b-41a0-b208-c964c45cdc68 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:47.818579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d2c5a-a14b-41a0-b208-c964c45cdc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:47.818854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d2c5a-a14b-41a0-b208-c964c45cdc68 None admin] PATH_INFO: `/v3/users/e7f4ca1f0d944803936e7e47b6f47d1d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:47.833172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d2c5a-a14b-41a0-b208-c964c45cdc68 None admin] RBAC: Authorizing `identity:get_user(user_id=e7f4ca1f0d944803936e7e47b6f47d1d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:47.834615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d2c5a-a14b-41a0-b208-c964c45cdc68 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:47.836221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 591/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:47 2022] GET /identity/v3/users/e7f4ca1f0d944803936e7e47b6f47d1d => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:47.842021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c097-0e03-4269-8e18-2e55165c2f03 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:47.873071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:47.940048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6629c097-0e03-4269-8e18-2e55165c2f03 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:47.973095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f867d2-cc53-4ff8-9985-a7783dcb7de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:47.973687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f867d2-cc53-4ff8-9985-a7783dcb7de0 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:47.973961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f867d2-cc53-4ff8-9985-a7783dcb7de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:47.974231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f867d2-cc53-4ff8-9985-a7783dcb7de0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:48.038062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f867d2-cc53-4ff8-9985-a7783dcb7de0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:48.039528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f867d2-cc53-4ff8-9985-a7783dcb7de0 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:48.068273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:48.134195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 592/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:02:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:02:48.140250 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d2c5a-a14b-41a0-b208-c964c45cdc68 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:48.237855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d2c5a-a14b-41a0-b208-c964c45cdc68 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:48.246653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:48.277968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128973d2-0ebf-4c58-b77c-ffe970cd0519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:48.278584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128973d2-0ebf-4c58-b77c-ffe970cd0519 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:48.278860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128973d2-0ebf-4c58-b77c-ffe970cd0519 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:48.279131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128973d2-0ebf-4c58-b77c-ffe970cd0519 None admin] PATH_INFO: `/v3/projects/7d50a8e2ba6645808363ab1661718271` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:48.280267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128973d2-0ebf-4c58-b77c-ffe970cd0519 None admin] RBAC: Authorizing `identity:get_project(project_id=7d50a8e2ba6645808363ab1661718271)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:48.281700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128973d2-0ebf-4c58-b77c-ffe970cd0519 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:48.284047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 592/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:48 2022] GET /identity/v3/projects/7d50a8e2ba6645808363ab1661718271 => generated 361 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:48.290329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f867d2-cc53-4ff8-9985-a7783dcb7de0 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:48.389780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f867d2-cc53-4ff8-9985-a7783dcb7de0 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:48.417824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:48.423030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1265f768-754a-4022-b0db-ae8999bb6b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:48.424332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265f768-754a-4022-b0db-ae8999bb6b4f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:48.424659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265f768-754a-4022-b0db-ae8999bb6b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:48.424923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1265f768-754a-4022-b0db-ae8999bb6b4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:48.425648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265f768-754a-4022-b0db-ae8999bb6b4f None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=7d50a8e2ba6645808363ab1661718271, user.id=e7f4ca1f0d944803936e7e47b6f47d1d) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:48.425955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265f768-754a-4022-b0db-ae8999bb6b4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:48.427672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1265f768-754a-4022-b0db-ae8999bb6b4f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:48.433129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 593/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:02:48 2022] GET /identity/v3/role_assignments?user.id=e7f4ca1f0d944803936e7e47b6f47d1d&scope.project.id=7d50a8e2ba6645808363ab1661718271&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:50.577418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3df99e-18bb-4bbb-8f51-435d24d363e6 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:50.577939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3df99e-18bb-4bbb-8f51-435d24d363e6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:50.577939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3df99e-18bb-4bbb-8f51-435d24d363e6 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:50.579280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 593/1186] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:50.584724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83b8bd-8748-4838-b7cf-a79741e82f72 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:50.585161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83b8bd-8748-4838-b7cf-a79741e82f72 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:50.585254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd83b8bd-8748-4838-b7cf-a79741e82f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:50.603732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-fd83b8bd-8748-4838-b7cf-a79741e82f72 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:50.604904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-fd83b8bd-8748-4838-b7cf-a79741e82f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:50.669324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 594/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:50.945285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3df99e-18bb-4bbb-8f51-435d24d363e6 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:51.045194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3df99e-18bb-4bbb-8f51-435d24d363e6 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:51.078233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c02df6-ebb3-49eb-a62f-f6ea1c59b62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:51.078843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c02df6-ebb3-49eb-a62f-f6ea1c59b62d None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:51.079120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c02df6-ebb3-49eb-a62f-f6ea1c59b62d None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:51.079390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c02df6-ebb3-49eb-a62f-f6ea1c59b62d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:51.090151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c02df6-ebb3-49eb-a62f-f6ea1c59b62d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-57c02df6-ebb3-49eb-a62f-f6ea1c59b62d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:02:51.094916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:51.096674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:02:51.097786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 594/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:02:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:51.102731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83b8bd-8748-4838-b7cf-a79741e82f72 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:51.196209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd83b8bd-8748-4838-b7cf-a79741e82f72 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:51.221673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:51.230331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933362a-59be-4d12-8ee8-b6de13fc7c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:51.230931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933362a-59be-4d12-8ee8-b6de13fc7c8f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:51.231207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933362a-59be-4d12-8ee8-b6de13fc7c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:51.231506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933362a-59be-4d12-8ee8-b6de13fc7c8f None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:51.232168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933362a-59be-4d12-8ee8-b6de13fc7c8f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:51.232468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933362a-59be-4d12-8ee8-b6de13fc7c8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:51.233831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933362a-59be-4d12-8ee8-b6de13fc7c8f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:51.241974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 595/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:02:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:51.247740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c02df6-ebb3-49eb-a62f-f6ea1c59b62d None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:51.283056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:51.348043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c02df6-ebb3-49eb-a62f-f6ea1c59b62d None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:51.381229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067de873-f864-4d30-a44d-ddb79f22b58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:51.381842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067de873-f864-4d30-a44d-ddb79f22b58b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:51.382115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067de873-f864-4d30-a44d-ddb79f22b58b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:51.382382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067de873-f864-4d30-a44d-ddb79f22b58b None admin] PATH_INFO: `/v3/users/e7f4ca1f0d944803936e7e47b6f47d1d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:51.383505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067de873-f864-4d30-a44d-ddb79f22b58b None admin] RBAC: Authorizing `identity:get_user(user_id=e7f4ca1f0d944803936e7e47b6f47d1d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:51.384945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067de873-f864-4d30-a44d-ddb79f22b58b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:51.386495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 595/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:51 2022] GET /identity/v3/users/e7f4ca1f0d944803936e7e47b6f47d1d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:51.393005 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933362a-59be-4d12-8ee8-b6de13fc7c8f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:51.490511 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933362a-59be-4d12-8ee8-b6de13fc7c8f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:51.523431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fddfa15-174b-44a3-9d07-1c99b9738bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:51.524066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddfa15-174b-44a3-9d07-1c99b9738bde None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:51.524338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddfa15-174b-44a3-9d07-1c99b9738bde None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:51.524604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fddfa15-174b-44a3-9d07-1c99b9738bde None admin] PATH_INFO: `/v3/projects/7d50a8e2ba6645808363ab1661718271` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:51.525687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddfa15-174b-44a3-9d07-1c99b9738bde None admin] RBAC: Authorizing `identity:get_project(project_id=7d50a8e2ba6645808363ab1661718271)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:51.527101 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fddfa15-174b-44a3-9d07-1c99b9738bde None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:51.528719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 596/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:51 2022] GET /identity/v3/projects/7d50a8e2ba6645808363ab1661718271 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:51.534797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067de873-f864-4d30-a44d-ddb79f22b58b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:51.631835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067de873-f864-4d30-a44d-ddb79f22b58b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:51.634092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:51.664557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f204f0-cf3b-47b7-a582-9e32826539d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:51.665191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f204f0-cf3b-47b7-a582-9e32826539d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:51.665463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f204f0-cf3b-47b7-a582-9e32826539d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:51.665732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f204f0-cf3b-47b7-a582-9e32826539d6 None admin] PATH_INFO: `/v3/projects/7d50a8e2ba6645808363ab1661718271/users/e7f4ca1f0d944803936e7e47b6f47d1d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:51.667612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f204f0-cf3b-47b7-a582-9e32826539d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d50a8e2ba6645808363ab1661718271, user_id=e7f4ca1f0d944803936e7e47b6f47d1d, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:51.669029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f204f0-cf3b-47b7-a582-9e32826539d6 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:51.681442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 596/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:02:51 2022] PUT /identity/v3/projects/7d50a8e2ba6645808363ab1661718271/users/e7f4ca1f0d944803936e7e47b6f47d1d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:02:53.791348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379febad-d654-4fe3-95f9-b9bd5d56838e None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:53.791888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379febad-d654-4fe3-95f9-b9bd5d56838e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:53.791888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379febad-d654-4fe3-95f9-b9bd5d56838e None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:53.793229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 597/1193] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:02:53.798322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0723e-5582-4265-b556-eb81775f3c3b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:53.799465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0723e-5582-4265-b556-eb81775f3c3b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:53.799465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0723e-5582-4265-b556-eb81775f3c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:53.817184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-35c0723e-5582-4265-b556-eb81775f3c3b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:02:53.818342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-35c0723e-5582-4265-b556-eb81775f3c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:02:53.882236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 597/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:02:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:02:54.149074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379febad-d654-4fe3-95f9-b9bd5d56838e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:54.204820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:54.248438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379febad-d654-4fe3-95f9-b9bd5d56838e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:54.280799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d87d44-50d2-47fb-8ae0-93f30f175bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:54.281413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d87d44-50d2-47fb-8ae0-93f30f175bd9 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:54.281691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d87d44-50d2-47fb-8ae0-93f30f175bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:54.281996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d87d44-50d2-47fb-8ae0-93f30f175bd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:54.292393 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d87d44-50d2-47fb-8ae0-93f30f175bd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-30d87d44-50d2-47fb-8ae0-93f30f175bd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:02:54.297134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:02:54.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:02:54.300047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 598/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:02:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:02:54.304846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0723e-5582-4265-b556-eb81775f3c3b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:54.395619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:54.402821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0723e-5582-4265-b556-eb81775f3c3b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:54.436726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d511381-3c67-40e9-8c82-ec02e36149ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:54.437332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d511381-3c67-40e9-8c82-ec02e36149ed None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:54.437961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d511381-3c67-40e9-8c82-ec02e36149ed None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:54.437961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d511381-3c67-40e9-8c82-ec02e36149ed None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:54.438508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d511381-3c67-40e9-8c82-ec02e36149ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:54.438803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d511381-3c67-40e9-8c82-ec02e36149ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:54.441065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d511381-3c67-40e9-8c82-ec02e36149ed None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:54.449351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 598/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:02:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:54.455823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d87d44-50d2-47fb-8ae0-93f30f175bd9 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:54.551856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d87d44-50d2-47fb-8ae0-93f30f175bd9 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:54.568062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:54.586080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0fe24-b9b3-4348-9905-5a27f6696e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:54.586690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0fe24-b9b3-4348-9905-5a27f6696e10 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:54.587019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0fe24-b9b3-4348-9905-5a27f6696e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:54.587237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c0fe24-b9b3-4348-9905-5a27f6696e10 None admin] PATH_INFO: `/v3/users/e7f4ca1f0d944803936e7e47b6f47d1d` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:54.588366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0fe24-b9b3-4348-9905-5a27f6696e10 None admin] RBAC: Authorizing `identity:get_user(user_id=e7f4ca1f0d944803936e7e47b6f47d1d)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:54.589796 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c0fe24-b9b3-4348-9905-5a27f6696e10 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:54.591355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 599/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Sat Sep 10 14:02:54 2022] GET /identity/v3/users/e7f4ca1f0d944803936e7e47b6f47d1d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:54.597689 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d511381-3c67-40e9-8c82-ec02e36149ed None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:54.695287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d511381-3c67-40e9-8c82-ec02e36149ed None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:54.727415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0f82-3151-4366-b95f-c1dd43b55ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:54.728024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0f82-3151-4366-b95f-c1dd43b55ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:54.728294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0f82-3151-4366-b95f-c1dd43b55ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:54.728559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b0f82-3151-4366-b95f-c1dd43b55ac7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:54.753298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:54.795536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0f82-3151-4366-b95f-c1dd43b55ac7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:54.796975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b0f82-3151-4366-b95f-c1dd43b55ac7 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:54.894013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 599/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:02:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:02:54.899965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0fe24-b9b3-4348-9905-5a27f6696e10 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:54.995495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c0fe24-b9b3-4348-9905-5a27f6696e10 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:55.028165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d74b2a-f7b7-4207-9f65-e86cd704b469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:55.028773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d74b2a-f7b7-4207-9f65-e86cd704b469 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:55.029047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d74b2a-f7b7-4207-9f65-e86cd704b469 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:55.029315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d74b2a-f7b7-4207-9f65-e86cd704b469 None admin] PATH_INFO: `/v3/projects/7d50a8e2ba6645808363ab1661718271` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:55.030397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d74b2a-f7b7-4207-9f65-e86cd704b469 None admin] RBAC: Authorizing `identity:get_project(project_id=7d50a8e2ba6645808363ab1661718271)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:55.031839 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d74b2a-f7b7-4207-9f65-e86cd704b469 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:55.033718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 600/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Sat Sep 10 14:02:54 2022] GET /identity/v3/projects/7d50a8e2ba6645808363ab1661718271 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:02:55.039718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0f82-3151-4366-b95f-c1dd43b55ac7 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:02:55.094104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:02:55.137586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b0f82-3151-4366-b95f-c1dd43b55ac7 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:02:55.171046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73c779-63f4-4200-841e-b39d23a593b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:02:55.171662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73c779-63f4-4200-841e-b39d23a593b1 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:02:55.171942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73c779-63f4-4200-841e-b39d23a593b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:02:55.172212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73c779-63f4-4200-841e-b39d23a593b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:02:55.172924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73c779-63f4-4200-841e-b39d23a593b1 None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=7d50a8e2ba6645808363ab1661718271, user.id=e7f4ca1f0d944803936e7e47b6f47d1d) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:02:55.173230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73c779-63f4-4200-841e-b39d23a593b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:02:55.175209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73c779-63f4-4200-841e-b39d23a593b1 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:02:55.180843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 600/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:02:55 2022] GET /identity/v3/role_assignments?user.id=e7f4ca1f0d944803936e7e47b6f47d1d&scope.project.id=7d50a8e2ba6645808363ab1661718271&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:03:05.024269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=74 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:05.178318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=40 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:51.261149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb961-488f-43ef-86ab-0b61ffad4d55 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:51.261735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb961-488f-43ef-86ab-0b61ffad4d55 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:51.261735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefb961-488f-43ef-86ab-0b61ffad4d55 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:51.263041 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 601/1201] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:03:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:03:51.270116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356bc5b-00b9-4e3f-b82f-5135fb7e0240 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:51.270615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356bc5b-00b9-4e3f-b82f-5135fb7e0240 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:51.270714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b356bc5b-00b9-4e3f-b82f-5135fb7e0240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:51.296059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b356bc5b-00b9-4e3f-b82f-5135fb7e0240 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:03:51.297795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b356bc5b-00b9-4e3f-b82f-5135fb7e0240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:03:51.395611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 601/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:03:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:03:51.687489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb961-488f-43ef-86ab-0b61ffad4d55 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:51.827058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefb961-488f-43ef-86ab-0b61ffad4d55 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:51.879028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4e65f-b4ad-4972-9a88-457475e899ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:51.880075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4e65f-b4ad-4972-9a88-457475e899ea None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:51.880352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4e65f-b4ad-4972-9a88-457475e899ea None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:51.880618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4e65f-b4ad-4972-9a88-457475e899ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:51.892708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4e65f-b4ad-4972-9a88-457475e899ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:51.894340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf4e65f-b4ad-4972-9a88-457475e899ea None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-5bf4e65f-b4ad-4972-9a88-457475e899ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:03:51.911667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:03:51.913025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:03:51.914238 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:03:51.914238 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:03:51.914238 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:03:51.915538 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 602/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:03:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:03:51.921983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356bc5b-00b9-4e3f-b82f-5135fb7e0240 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:52.064417 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b356bc5b-00b9-4e3f-b82f-5135fb7e0240 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:52.102671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:52.109127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eeb399-3a07-47f0-a7ca-f61c8a15f7d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:52.109791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeb399-3a07-47f0-a7ca-f61c8a15f7d5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:52.110144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeb399-3a07-47f0-a7ca-f61c8a15f7d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:52.110577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eeb399-3a07-47f0-a7ca-f61c8a15f7d5 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:52.111252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeb399-3a07-47f0-a7ca-f61c8a15f7d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:03:52.111686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeb399-3a07-47f0-a7ca-f61c8a15f7d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:52.113232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eeb399-3a07-47f0-a7ca-f61c8a15f7d5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:52.132562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 602/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:03:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:03:52.138732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4e65f-b4ad-4972-9a88-457475e899ea None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:52.213753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:52.290193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf4e65f-b4ad-4972-9a88-457475e899ea None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:52.330311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69a1c5e-fec8-4ce5-a7d9-f6e0a3ad49b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:52.331003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a1c5e-fec8-4ce5-a7d9-f6e0a3ad49b1 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:52.331617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a1c5e-fec8-4ce5-a7d9-f6e0a3ad49b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:52.332012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69a1c5e-fec8-4ce5-a7d9-f6e0a3ad49b1 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:52.332740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a1c5e-fec8-4ce5-a7d9-f6e0a3ad49b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:52.334368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69a1c5e-fec8-4ce5-a7d9-f6e0a3ad49b1 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:52.360045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 603/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Sat Sep 10 14:03:52 2022] POST /identity/v3/users => generated 251 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:03:54.780773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb05064-446f-483d-96ea-099bc5a24d93 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:54.781290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb05064-446f-483d-96ea-099bc5a24d93 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:54.781290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb05064-446f-483d-96ea-099bc5a24d93 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:54.785252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 603/1206] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:03:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:03:54.788631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3950cfa-6116-486a-97c1-93ff81a9e5cb None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:54.789102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3950cfa-6116-486a-97c1-93ff81a9e5cb None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:54.789197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3950cfa-6116-486a-97c1-93ff81a9e5cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:54.808370 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a3950cfa-6116-486a-97c1-93ff81a9e5cb None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:03:54.809568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a3950cfa-6116-486a-97c1-93ff81a9e5cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:03:54.888276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 604/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:03:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 10 14:03:55.167489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb05064-446f-483d-96ea-099bc5a24d93 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:55.288726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb05064-446f-483d-96ea-099bc5a24d93 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:55.350781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74f4e2-c384-4fdc-baeb-75fc7b1e0455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:55.351431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74f4e2-c384-4fdc-baeb-75fc7b1e0455 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:55.351769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74f4e2-c384-4fdc-baeb-75fc7b1e0455 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:55.351994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc74f4e2-c384-4fdc-baeb-75fc7b1e0455 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:55.364267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc74f4e2-c384-4fdc-baeb-75fc7b1e0455 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-dc74f4e2-c384-4fdc-baeb-75fc7b1e0455 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:03:55.369831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:03:55.370846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:03:55.378536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 604/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:03:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:03:55.379268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3950cfa-6116-486a-97c1-93ff81a9e5cb None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:55.430834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:55.508442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3950cfa-6116-486a-97c1-93ff81a9e5cb None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:55.551906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2591865-e3c1-450c-ad8e-5f2bef37971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:55.552545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2591865-e3c1-450c-ad8e-5f2bef37971f None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:55.552849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2591865-e3c1-450c-ad8e-5f2bef37971f None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:55.553178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2591865-e3c1-450c-ad8e-5f2bef37971f None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:55.554148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2591865-e3c1-450c-ad8e-5f2bef37971f None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:03:55.554458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2591865-e3c1-450c-ad8e-5f2bef37971f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:55.555919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2591865-e3c1-450c-ad8e-5f2bef37971f None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:55.567470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 605/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:03:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:03:55.573501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74f4e2-c384-4fdc-baeb-75fc7b1e0455 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:55.645423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:55.715905 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc74f4e2-c384-4fdc-baeb-75fc7b1e0455 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:55.757059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de530dde-cf31-4469-baaf-09716f2741ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:55.757933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de530dde-cf31-4469-baaf-09716f2741ac None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:55.758252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de530dde-cf31-4469-baaf-09716f2741ac None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:55.758520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de530dde-cf31-4469-baaf-09716f2741ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:55.771998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de530dde-cf31-4469-baaf-09716f2741ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:55.773986 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de530dde-cf31-4469-baaf-09716f2741ac None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-de530dde-cf31-4469-baaf-09716f2741ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:03:55.790531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:03:55.794280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:03:55.795177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:03:55.795177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:03:55.795177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:03:55.795177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 605/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:03:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:03:55.799164 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2591865-e3c1-450c-ad8e-5f2bef37971f None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:55.926152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:55.929350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2591865-e3c1-450c-ad8e-5f2bef37971f None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:55.975903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c6b21-33f8-4c89-be2e-44dde4856b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:55.976424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c6b21-33f8-4c89-be2e-44dde4856b9c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:55.976754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c6b21-33f8-4c89-be2e-44dde4856b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:55.977078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c6b21-33f8-4c89-be2e-44dde4856b9c None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:55.977696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c6b21-33f8-4c89-be2e-44dde4856b9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:03:55.978078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c6b21-33f8-4c89-be2e-44dde4856b9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:55.979596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c6b21-33f8-4c89-be2e-44dde4856b9c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:55.993454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 606/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:03:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:03:55.999582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de530dde-cf31-4469-baaf-09716f2741ac None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:56.113085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de530dde-cf31-4469-baaf-09716f2741ac None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:56.115224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:56.153745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063b06a-c2a9-4911-b9a6-905bde5859fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:56.154449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063b06a-c2a9-4911-b9a6-905bde5859fe None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:56.154762 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063b06a-c2a9-4911-b9a6-905bde5859fe None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:56.155122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d063b06a-c2a9-4911-b9a6-905bde5859fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:56.165588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063b06a-c2a9-4911-b9a6-905bde5859fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:56.167299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d063b06a-c2a9-4911-b9a6-905bde5859fe None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d063b06a-c2a9-4911-b9a6-905bde5859fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:03:56.183264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:03:56.184496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:03:56.185461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 606/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:03:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:03:56.191769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c6b21-33f8-4c89-be2e-44dde4856b9c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:56.299488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c6b21-33f8-4c89-be2e-44dde4856b9c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:56.338797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd2595-fed6-4e9c-a21b-ef85ab530dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:56.339540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd2595-fed6-4e9c-a21b-ef85ab530dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:56.339849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd2595-fed6-4e9c-a21b-ef85ab530dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:56.340125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccd2595-fed6-4e9c-a21b-ef85ab530dc1 None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:56.340888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd2595-fed6-4e9c-a21b-ef85ab530dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:03:56.341198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd2595-fed6-4e9c-a21b-ef85ab530dc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:56.342701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccd2595-fed6-4e9c-a21b-ef85ab530dc1 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:56.364723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 607/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 10 14:03:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:03:56.370707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063b06a-c2a9-4911-b9a6-905bde5859fe None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:56.490909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d063b06a-c2a9-4911-b9a6-905bde5859fe None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:56.534600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5522a8e-e987-48b6-aad5-d02c3985cfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:56.535392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5522a8e-e987-48b6-aad5-d02c3985cfc8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:56.535727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5522a8e-e987-48b6-aad5-d02c3985cfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:56.536004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5522a8e-e987-48b6-aad5-d02c3985cfc8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:56.573295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:56.624034 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5522a8e-e987-48b6-aad5-d02c3985cfc8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:56.625763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5522a8e-e987-48b6-aad5-d02c3985cfc8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:56.746698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 607/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:03:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:03:56.748652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd2595-fed6-4e9c-a21b-ef85ab530dc1 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:56.772335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:56.880958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccd2595-fed6-4e9c-a21b-ef85ab530dc1 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:56.922945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80120a-5e63-4987-9400-f4554e701490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:56.923680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80120a-5e63-4987-9400-f4554e701490 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:56.923973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80120a-5e63-4987-9400-f4554e701490 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:56.924303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80120a-5e63-4987-9400-f4554e701490 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:56.935131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80120a-5e63-4987-9400-f4554e701490 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:56.936902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80120a-5e63-4987-9400-f4554e701490 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-cc80120a-5e63-4987-9400-f4554e701490 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:03:56.957711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:03:56.958782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:03:56.964779 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:03:56.964779 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:03:56.964779 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:03:56.964779 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 608/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:03:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:03:56.965078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5522a8e-e987-48b6-aad5-d02c3985cfc8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:57.038985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:57.086479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5522a8e-e987-48b6-aad5-d02c3985cfc8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:57.123587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b80558-0815-4638-8735-a297c5777d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:57.124120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b80558-0815-4638-8735-a297c5777d5e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:57.124395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b80558-0815-4638-8735-a297c5777d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:57.124814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b80558-0815-4638-8735-a297c5777d5e None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:57.126037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b80558-0815-4638-8735-a297c5777d5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:03:57.126303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b80558-0815-4638-8735-a297c5777d5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:57.127731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b80558-0815-4638-8735-a297c5777d5e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:57.145769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 608/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:03:57.146972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80120a-5e63-4987-9400-f4554e701490 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:57.227899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:57.263238 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80120a-5e63-4987-9400-f4554e701490 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:57.305186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b788-7cd6-4308-bdc1-75cec472fc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:57.305820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b788-7cd6-4308-bdc1-75cec472fc4d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:57.306098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b788-7cd6-4308-bdc1-75cec472fc4d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:57.306366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b4b788-7cd6-4308-bdc1-75cec472fc4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:57.315516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4b788-7cd6-4308-bdc1-75cec472fc4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:57.317055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b4b788-7cd6-4308-bdc1-75cec472fc4d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-d4b4b788-7cd6-4308-bdc1-75cec472fc4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:03:57.329756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:03:57.330701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:03:57.331682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:03:57.331682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:03:57.331682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:03:57.331682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:03:57.331682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:03:57.331682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:03:57.331682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:03:57.331682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:03:57.335963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 609/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:03:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:03:57.339041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b80558-0815-4638-8735-a297c5777d5e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:57.467236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b80558-0815-4638-8735-a297c5777d5e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:57.494944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:57.510918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0e0cf-dec8-4b10-91ae-7d0102183848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:57.511565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0e0cf-dec8-4b10-91ae-7d0102183848 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:57.512003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0e0cf-dec8-4b10-91ae-7d0102183848 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:57.512436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0e0cf-dec8-4b10-91ae-7d0102183848 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:57.513303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0e0cf-dec8-4b10-91ae-7d0102183848 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:03:57.513681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0e0cf-dec8-4b10-91ae-7d0102183848 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:57.515303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0e0cf-dec8-4b10-91ae-7d0102183848 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:57.538235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b788-7cd6-4308-bdc1-75cec472fc4d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:03:57.539481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 609/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:03:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:03:57.648797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b4b788-7cd6-4308-bdc1-75cec472fc4d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:03:57.687563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:03:57.696299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c618a-7670-43c7-8ccb-a7dc7f2b8b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:03:57.696299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c618a-7670-43c7-8ccb-a7dc7f2b8b60 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:03:57.700564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c618a-7670-43c7-8ccb-a7dc7f2b8b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:03:57.700564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c618a-7670-43c7-8ccb-a7dc7f2b8b60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:03:57.700564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c618a-7670-43c7-8ccb-a7dc7f2b8b60 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=2a6a70a10cfa49348cc8b2b42870909d) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:03:57.700564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c618a-7670-43c7-8ccb-a7dc7f2b8b60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:03:57.700564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c618a-7670-43c7-8ccb-a7dc7f2b8b60 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:03:57.706186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 610/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:03:57 2022] GET /identity/v3/role_assignments?user.id=2a6a70a10cfa49348cc8b2b42870909d&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:00.005722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c9c8-c980-4b1c-a025-94c74ac85f1e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:00.006653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c9c8-c980-4b1c-a025-94c74ac85f1e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:00.006653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de7c9c8-c980-4b1c-a025-94c74ac85f1e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:00.007629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 610/1220] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:04:00.014465 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834c010-5355-4646-8376-29f5c3ae592d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:00.014931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834c010-5355-4646-8376-29f5c3ae592d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:00.015272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834c010-5355-4646-8376-29f5c3ae592d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:00.035571 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e834c010-5355-4646-8376-29f5c3ae592d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:04:00.036782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e834c010-5355-4646-8376-29f5c3ae592d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:04:00.108600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 611/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:04:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:04:00.380448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7c9c8-c980-4b1c-a025-94c74ac85f1e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:00.497055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de7c9c8-c980-4b1c-a025-94c74ac85f1e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:00.534904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260d389-8fcc-48d6-a039-ffca126f70c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:00.535567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260d389-8fcc-48d6-a039-ffca126f70c0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:00.535854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260d389-8fcc-48d6-a039-ffca126f70c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:00.536119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260d389-8fcc-48d6-a039-ffca126f70c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:00.549590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260d389-8fcc-48d6-a039-ffca126f70c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-8260d389-8fcc-48d6-a039-ffca126f70c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:04:00.557180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:00.559704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:00.560830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 611/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:04:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:00.564676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834c010-5355-4646-8376-29f5c3ae592d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:00.671486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834c010-5355-4646-8376-29f5c3ae592d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:00.697838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:00.710176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5deba-7072-4ca4-925a-9cdd31836e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:00.710845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5deba-7072-4ca4-925a-9cdd31836e2b None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:00.711123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5deba-7072-4ca4-925a-9cdd31836e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:00.711424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd5deba-7072-4ca4-925a-9cdd31836e2b None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:00.712176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5deba-7072-4ca4-925a-9cdd31836e2b None admin] RBAC: Adding query filter params (name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:00.712496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5deba-7072-4ca4-925a-9cdd31836e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:00.714146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd5deba-7072-4ca4-925a-9cdd31836e2b None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:00.724030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 612/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:04:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:00.730680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260d389-8fcc-48d6-a039-ffca126f70c0 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:00.779671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:00.842060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260d389-8fcc-48d6-a039-ffca126f70c0 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:00.881643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a94a0-a9d8-413e-87cf-4556b2d6a9e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:00.882323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a94a0-a9d8-413e-87cf-4556b2d6a9e5 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:00.882610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a94a0-a9d8-413e-87cf-4556b2d6a9e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:00.882872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4a94a0-a9d8-413e-87cf-4556b2d6a9e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:00.894456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a94a0-a9d8-413e-87cf-4556b2d6a9e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:00.895996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4a94a0-a9d8-413e-87cf-4556b2d6a9e5 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-af4a94a0-a9d8-413e-87cf-4556b2d6a9e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:04:00.911290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:00.917156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:04:00.918180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:04:00.918180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:00.918180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:00.918180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 612/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:00.919638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5deba-7072-4ca4-925a-9cdd31836e2b None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:01.031545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd5deba-7072-4ca4-925a-9cdd31836e2b None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:01.070357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddab6e-95e9-476c-b278-6f456df728dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:01.071097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddab6e-95e9-476c-b278-6f456df728dc None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:01.071381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddab6e-95e9-476c-b278-6f456df728dc None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:01.071690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddab6e-95e9-476c-b278-6f456df728dc None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:01.072320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddab6e-95e9-476c-b278-6f456df728dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:01.072648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddab6e-95e9-476c-b278-6f456df728dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:01.074284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ddab6e-95e9-476c-b278-6f456df728dc None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:01.087217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 613/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:01.092992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a94a0-a9d8-413e-87cf-4556b2d6a9e5 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:01.204195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:01.211921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4a94a0-a9d8-413e-87cf-4556b2d6a9e5 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:01.246780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a1603-da28-4788-91b9-98cd876ba757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:01.247561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a1603-da28-4788-91b9-98cd876ba757 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:01.247827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a1603-da28-4788-91b9-98cd876ba757 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:01.248267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66a1603-da28-4788-91b9-98cd876ba757 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:01.257026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a1603-da28-4788-91b9-98cd876ba757 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:01.258498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66a1603-da28-4788-91b9-98cd876ba757 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-b66a1603-da28-4788-91b9-98cd876ba757 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:04:01.271671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:01.272926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:01.273982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 613/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:01.280277 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddab6e-95e9-476c-b278-6f456df728dc None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:01.331429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:01.394179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddab6e-95e9-476c-b278-6f456df728dc None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:01.432080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df29c43-b938-410c-b465-364ca268f79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:01.432738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df29c43-b938-410c-b465-364ca268f79c None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:01.433054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df29c43-b938-410c-b465-364ca268f79c None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:01.433344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df29c43-b938-410c-b465-364ca268f79c None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:01.434075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df29c43-b938-410c-b465-364ca268f79c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:01.434382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df29c43-b938-410c-b465-364ca268f79c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:01.435965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df29c43-b938-410c-b465-364ca268f79c None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:01.456485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 614/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 10 14:04:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:04:01.462269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a1603-da28-4788-91b9-98cd876ba757 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:01.579693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66a1603-da28-4788-91b9-98cd876ba757 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:01.615652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad2558-6dc7-45ce-a39d-72801f2c8313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:01.616200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad2558-6dc7-45ce-a39d-72801f2c8313 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:01.616475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad2558-6dc7-45ce-a39d-72801f2c8313 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:01.616740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad2558-6dc7-45ce-a39d-72801f2c8313 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:01.627603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad2558-6dc7-45ce-a39d-72801f2c8313 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:01.629050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad2558-6dc7-45ce-a39d-72801f2c8313 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-59ad2558-6dc7-45ce-a39d-72801f2c8313 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:04:01.645133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:01.646124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:04:01.647043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:04:01.647043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:01.647043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:01.647043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 614/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:01.652322 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df29c43-b938-410c-b465-364ca268f79c None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:01.764961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df29c43-b938-410c-b465-364ca268f79c None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:01.767980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:01.804287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29224-73d1-4f75-af02-85fcc6bcc746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:01.804869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29224-73d1-4f75-af02-85fcc6bcc746 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:01.805173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29224-73d1-4f75-af02-85fcc6bcc746 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:01.805557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb29224-73d1-4f75-af02-85fcc6bcc746 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:01.806083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb29224-73d1-4f75-af02-85fcc6bcc746 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:01.806387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb29224-73d1-4f75-af02-85fcc6bcc746 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:01.807828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb29224-73d1-4f75-af02-85fcc6bcc746 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:01.819728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 615/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:04:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:04:01.825697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad2558-6dc7-45ce-a39d-72801f2c8313 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:01.836460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:01.933373 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad2558-6dc7-45ce-a39d-72801f2c8313 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:01.975855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90567031-6039-4f7b-826d-2f74d624a185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:01.976496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90567031-6039-4f7b-826d-2f74d624a185 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:01.976831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90567031-6039-4f7b-826d-2f74d624a185 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:01.977103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90567031-6039-4f7b-826d-2f74d624a185 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:01.984818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90567031-6039-4f7b-826d-2f74d624a185 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:01.986401 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90567031-6039-4f7b-826d-2f74d624a185 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-90567031-6039-4f7b-826d-2f74d624a185 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:04:01.998398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:01.999659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:02.001662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:04:02.001662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:04:02.001662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:04:02.001662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:04:02.001662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:04:02.001662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:04:02.001662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:04:02.001662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:02.001662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 615/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:04:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:02.006279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29224-73d1-4f75-af02-85fcc6bcc746 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:02.116803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb29224-73d1-4f75-af02-85fcc6bcc746 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:02.152384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8cf00-536d-42ac-90c0-5dbe589c2a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:02.153012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8cf00-536d-42ac-90c0-5dbe589c2a27 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:02.153444 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8cf00-536d-42ac-90c0-5dbe589c2a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:02.153779 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8cf00-536d-42ac-90c0-5dbe589c2a27 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:02.154538 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8cf00-536d-42ac-90c0-5dbe589c2a27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:02.154833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8cf00-536d-42ac-90c0-5dbe589c2a27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:02.157019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8cf00-536d-42ac-90c0-5dbe589c2a27 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:02.170136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 616/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:04:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:02.177298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90567031-6039-4f7b-826d-2f74d624a185 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:02.249799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:02.293869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90567031-6039-4f7b-826d-2f74d624a185 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:02.333977 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63520679-4f17-48df-ac57-00baa95aeadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:02.334789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63520679-4f17-48df-ac57-00baa95aeadd None admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:02.335070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63520679-4f17-48df-ac57-00baa95aeadd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:02.335347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63520679-4f17-48df-ac57-00baa95aeadd None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/2a6a70a10cfa49348cc8b2b42870909d/roles/a24499651a7b4225ac5459b8b59f1e24` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:02.351697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63520679-4f17-48df-ac57-00baa95aeadd None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=2a6a70a10cfa49348cc8b2b42870909d, role_id=a24499651a7b4225ac5459b8b59f1e24)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:02.353499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63520679-4f17-48df-ac57-00baa95aeadd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:02.367637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 616/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:04:02 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/2a6a70a10cfa49348cc8b2b42870909d/roles/a24499651a7b4225ac5459b8b59f1e24 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:04:04.623965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddf6b7-38fb-40fd-a796-b3f8905b6ee6 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:04.624513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddf6b7-38fb-40fd-a796-b3f8905b6ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:04.624645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddf6b7-38fb-40fd-a796-b3f8905b6ee6 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:04.626215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 617/1233] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:04:04.632084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edd3a9-d227-4759-bc44-6ebeb3080380 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:04.632600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edd3a9-d227-4759-bc44-6ebeb3080380 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:04.632706 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edd3a9-d227-4759-bc44-6ebeb3080380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:04.652912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-41edd3a9-d227-4759-bc44-6ebeb3080380 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:04:04.654123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-41edd3a9-d227-4759-bc44-6ebeb3080380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:04:04.733069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 617/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:04:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:04:05.038639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddf6b7-38fb-40fd-a796-b3f8905b6ee6 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:05.065472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:05.150651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddf6b7-38fb-40fd-a796-b3f8905b6ee6 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:05.187594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6215b-3c4b-48da-9311-62dc6769ec49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:05.188515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6215b-3c4b-48da-9311-62dc6769ec49 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:05.188789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6215b-3c4b-48da-9311-62dc6769ec49 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:05.189059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae6215b-3c4b-48da-9311-62dc6769ec49 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:05.199603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae6215b-3c4b-48da-9311-62dc6769ec49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-7ae6215b-3c4b-48da-9311-62dc6769ec49 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:04:05.204663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:05.205802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:05.206793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 618/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Sat Sep 10 14:04:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:05.212426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edd3a9-d227-4759-bc44-6ebeb3080380 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:05.260281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:05.317936 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edd3a9-d227-4759-bc44-6ebeb3080380 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:05.354726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007490c8-aecb-48dc-805f-a36fe401832a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:05.355345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007490c8-aecb-48dc-805f-a36fe401832a None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:05.355734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007490c8-aecb-48dc-805f-a36fe401832a None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:05.356025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007490c8-aecb-48dc-805f-a36fe401832a None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:05.356780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007490c8-aecb-48dc-805f-a36fe401832a None admin] RBAC: Adding query filter params (name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:05.357109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007490c8-aecb-48dc-805f-a36fe401832a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:05.358674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007490c8-aecb-48dc-805f-a36fe401832a None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:05.374514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 618/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Sat Sep 10 14:04:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:04:05.374712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6215b-3c4b-48da-9311-62dc6769ec49 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:05.470863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:05.489733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae6215b-3c4b-48da-9311-62dc6769ec49 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:05.527324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174368bd-dc73-4f2c-91dd-000dabbeb851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:05.528001 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174368bd-dc73-4f2c-91dd-000dabbeb851 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:05.528261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174368bd-dc73-4f2c-91dd-000dabbeb851 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:05.528531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174368bd-dc73-4f2c-91dd-000dabbeb851 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:05.539152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174368bd-dc73-4f2c-91dd-000dabbeb851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:05.540687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174368bd-dc73-4f2c-91dd-000dabbeb851 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-174368bd-dc73-4f2c-91dd-000dabbeb851 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:04:05.557824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:05.559056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:04:05.560373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:04:05.560373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:05.560373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:05.560373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 619/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:04:05.565560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007490c8-aecb-48dc-805f-a36fe401832a None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:05.678259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007490c8-aecb-48dc-805f-a36fe401832a None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:05.687798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:05.716304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eef611-c917-423d-ac3e-028716d2c0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:05.716888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eef611-c917-423d-ac3e-028716d2c0c8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:05.717164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eef611-c917-423d-ac3e-028716d2c0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:05.717448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8eef611-c917-423d-ac3e-028716d2c0c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:05.718090 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eef611-c917-423d-ac3e-028716d2c0c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:05.718370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eef611-c917-423d-ac3e-028716d2c0c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:05.720387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8eef611-c917-423d-ac3e-028716d2c0c8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:05.738012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 619/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:05.739575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174368bd-dc73-4f2c-91dd-000dabbeb851 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:05.855676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174368bd-dc73-4f2c-91dd-000dabbeb851 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:05.892304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bd8c3-d6f3-43dc-843e-f32dad47d4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:05.892930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bd8c3-d6f3-43dc-843e-f32dad47d4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:05.893205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bd8c3-d6f3-43dc-843e-f32dad47d4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:05.893480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613bd8c3-d6f3-43dc-843e-f32dad47d4e6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:05.896163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:05.901743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bd8c3-d6f3-43dc-843e-f32dad47d4e6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:05.903468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613bd8c3-d6f3-43dc-843e-f32dad47d4e6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-613bd8c3-d6f3-43dc-843e-f32dad47d4e6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:04:05.915898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:05.917189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:05.918148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 620/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:05.923666 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eef611-c917-423d-ac3e-028716d2c0c8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:06.035798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8eef611-c917-423d-ac3e-028716d2c0c8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:06.076665 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8f885-e10e-4076-9be4-4b8db9ee6cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:06.077256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8f885-e10e-4076-9be4-4b8db9ee6cb8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:06.077529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8f885-e10e-4076-9be4-4b8db9ee6cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:06.077796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8f885-e10e-4076-9be4-4b8db9ee6cb8 None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:06.078844 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8f885-e10e-4076-9be4-4b8db9ee6cb8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:06.079148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8f885-e10e-4076-9be4-4b8db9ee6cb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:06.080627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8f885-e10e-4076-9be4-4b8db9ee6cb8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:06.098919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 620/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 10 14:04:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:06.105019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bd8c3-d6f3-43dc-843e-f32dad47d4e6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:06.220602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613bd8c3-d6f3-43dc-843e-f32dad47d4e6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:06.257108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d4d53-6bbb-45da-915d-023b2b2ea033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:06.257848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d4d53-6bbb-45da-915d-023b2b2ea033 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:06.258200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d4d53-6bbb-45da-915d-023b2b2ea033 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:06.258514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d4d53-6bbb-45da-915d-023b2b2ea033 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:06.327113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:06.335500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d4d53-6bbb-45da-915d-023b2b2ea033 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:06.336987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d4d53-6bbb-45da-915d-023b2b2ea033 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:06.449994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 621/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:04:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:04:06.456641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8f885-e10e-4076-9be4-4b8db9ee6cb8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:06.489654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:06.572911 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8f885-e10e-4076-9be4-4b8db9ee6cb8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:06.612705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61d95c-3950-43d5-9cee-694e73a81839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:06.613253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61d95c-3950-43d5-9cee-694e73a81839 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:06.613526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61d95c-3950-43d5-9cee-694e73a81839 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:06.613800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e61d95c-3950-43d5-9cee-694e73a81839 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:06.625430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61d95c-3950-43d5-9cee-694e73a81839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:06.627311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e61d95c-3950-43d5-9cee-694e73a81839 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9e61d95c-3950-43d5-9cee-694e73a81839 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:04:06.644022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:06.645163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:04:06.651595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:04:06.651595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:06.651595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:06.651595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 621/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:04:06.652112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d4d53-6bbb-45da-915d-023b2b2ea033 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:06.752839 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:06.758786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d4d53-6bbb-45da-915d-023b2b2ea033 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:06.795930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a333d4-9784-4a80-968d-b908a85ff17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:06.796555 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a333d4-9784-4a80-968d-b908a85ff17a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:06.797650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a333d4-9784-4a80-968d-b908a85ff17a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:06.797956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a333d4-9784-4a80-968d-b908a85ff17a None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:06.798690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a333d4-9784-4a80-968d-b908a85ff17a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:06.798985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a333d4-9784-4a80-968d-b908a85ff17a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:06.800470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a333d4-9784-4a80-968d-b908a85ff17a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:06.813235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 622/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:06.818976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61d95c-3950-43d5-9cee-694e73a81839 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:06.908472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:06.930755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e61d95c-3950-43d5-9cee-694e73a81839 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:06.967158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a3278-923f-4825-bbd8-24b88feb176b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:06.967825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a3278-923f-4825-bbd8-24b88feb176b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:06.968102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a3278-923f-4825-bbd8-24b88feb176b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:06.968460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a3278-923f-4825-bbd8-24b88feb176b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:06.976162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a3278-923f-4825-bbd8-24b88feb176b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:06.977713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a3278-923f-4825-bbd8-24b88feb176b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-c24a3278-923f-4825-bbd8-24b88feb176b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:04:06.991204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:06.992477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:06.993552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:04:06.993552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:04:06.993552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:04:06.993552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:04:06.993552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:04:06.993552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:04:06.993552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:04:06.993552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:06.993552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 622/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:04:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:04:06.998729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a333d4-9784-4a80-968d-b908a85ff17a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:07.110491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a333d4-9784-4a80-968d-b908a85ff17a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:07.147227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6311b1b-2958-4b86-9350-e9a07014ee57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:07.147857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6311b1b-2958-4b86-9350-e9a07014ee57 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:07.148131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6311b1b-2958-4b86-9350-e9a07014ee57 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:07.148401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6311b1b-2958-4b86-9350-e9a07014ee57 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:07.149089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6311b1b-2958-4b86-9350-e9a07014ee57 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:07.149387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6311b1b-2958-4b86-9350-e9a07014ee57 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:07.150801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6311b1b-2958-4b86-9350-e9a07014ee57 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:07.168777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 623/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:04:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:07.170003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a3278-923f-4825-bbd8-24b88feb176b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:07.280588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a3278-923f-4825-bbd8-24b88feb176b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:07.317250 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180d5342-bfde-4066-82d9-9d166799f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:07.317899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d5342-bfde-4066-82d9-9d166799f697 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:07.318225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d5342-bfde-4066-82d9-9d166799f697 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:07.318513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180d5342-bfde-4066-82d9-9d166799f697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:07.319241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d5342-bfde-4066-82d9-9d166799f697 None admin] RBAC: Adding query filter params (role.id=a24499651a7b4225ac5459b8b59f1e24, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=2a6a70a10cfa49348cc8b2b42870909d) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:07.319579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d5342-bfde-4066-82d9-9d166799f697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:07.321093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180d5342-bfde-4066-82d9-9d166799f697 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:07.327029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 623/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:04:07 2022] GET /identity/v3/role_assignments?user.id=2a6a70a10cfa49348cc8b2b42870909d&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=a24499651a7b4225ac5459b8b59f1e24 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:04:09.799590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474de3e-60d7-4390-80e6-ab3f481865f5 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:09.800097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474de3e-60d7-4390-80e6-ab3f481865f5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:09.800097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474de3e-60d7-4390-80e6-ab3f481865f5 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:09.801768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 624/1247] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:04:09.809225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195129-c6dc-4c51-8a77-2f7273289c2f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:09.809701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195129-c6dc-4c51-8a77-2f7273289c2f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:09.809799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78195129-c6dc-4c51-8a77-2f7273289c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:09.817331 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:09.832521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-78195129-c6dc-4c51-8a77-2f7273289c2f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:04:09.834301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-78195129-c6dc-4c51-8a77-2f7273289c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:04:09.911263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 624/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:04:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:04:10.201581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474de3e-60d7-4390-80e6-ab3f481865f5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:10.220753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:10.316808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474de3e-60d7-4390-80e6-ab3f481865f5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:10.353322 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00ad2b-d03c-4927-930d-1d16f54c571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:10.353974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00ad2b-d03c-4927-930d-1d16f54c571d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:10.354253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00ad2b-d03c-4927-930d-1d16f54c571d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:10.354564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef00ad2b-d03c-4927-930d-1d16f54c571d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:10.366411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef00ad2b-d03c-4927-930d-1d16f54c571d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-ef00ad2b-d03c-4927-930d-1d16f54c571d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:04:10.371644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:10.372653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:10.376316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 625/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:04:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:10.380023 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195129-c6dc-4c51-8a77-2f7273289c2f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:10.492358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78195129-c6dc-4c51-8a77-2f7273289c2f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:10.524765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:10.531423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e1e74-44a3-4ce7-b55a-a3984040cbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:10.532236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1e74-44a3-4ce7-b55a-a3984040cbf4 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:10.532549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1e74-44a3-4ce7-b55a-a3984040cbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:10.533044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02e1e74-44a3-4ce7-b55a-a3984040cbf4 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:10.533814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e1e74-44a3-4ce7-b55a-a3984040cbf4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:10.534164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e1e74-44a3-4ce7-b55a-a3984040cbf4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:10.535663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02e1e74-44a3-4ce7-b55a-a3984040cbf4 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:10.550334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 625/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:04:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:10.551606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00ad2b-d03c-4927-930d-1d16f54c571d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:10.638407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:10.662528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef00ad2b-d03c-4927-930d-1d16f54c571d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:10.698850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d550ec9-ed4b-461a-a115-0fdbde0167e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:10.699521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d550ec9-ed4b-461a-a115-0fdbde0167e6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:10.699833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d550ec9-ed4b-461a-a115-0fdbde0167e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:10.700106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d550ec9-ed4b-461a-a115-0fdbde0167e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:10.711725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d550ec9-ed4b-461a-a115-0fdbde0167e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:10.713201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d550ec9-ed4b-461a-a115-0fdbde0167e6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-9d550ec9-ed4b-461a-a115-0fdbde0167e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:04:10.730399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:10.731658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:04:10.732771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:04:10.732771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:10.732771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:10.732771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 626/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 10 14:04:10.739917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e1e74-44a3-4ce7-b55a-a3984040cbf4 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:10.862072 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02e1e74-44a3-4ce7-b55a-a3984040cbf4 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:10.897965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cb83b-3d2a-450f-ba20-c5c5d82599ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:10.898579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cb83b-3d2a-450f-ba20-c5c5d82599ba None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:10.898850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cb83b-3d2a-450f-ba20-c5c5d82599ba None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:10.899113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90cb83b-3d2a-450f-ba20-c5c5d82599ba None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:10.899806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90cb83b-3d2a-450f-ba20-c5c5d82599ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:10.900105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90cb83b-3d2a-450f-ba20-c5c5d82599ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:10.901508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90cb83b-3d2a-450f-ba20-c5c5d82599ba None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:10.914661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 626/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:10.920547 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d550ec9-ed4b-461a-a115-0fdbde0167e6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:11.023276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d550ec9-ed4b-461a-a115-0fdbde0167e6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:11.049693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:11.057968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84403656-84a7-4eb4-abe7-1067d6b6373d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:11.058601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84403656-84a7-4eb4-abe7-1067d6b6373d None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:11.058879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84403656-84a7-4eb4-abe7-1067d6b6373d None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:11.059149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84403656-84a7-4eb4-abe7-1067d6b6373d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:11.067540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84403656-84a7-4eb4-abe7-1067d6b6373d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:11.068932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84403656-84a7-4eb4-abe7-1067d6b6373d None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-84403656-84a7-4eb4-abe7-1067d6b6373d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:04:11.080212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:11.081347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:11.082460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 627/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:11.087207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cb83b-3d2a-450f-ba20-c5c5d82599ba None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:11.144082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:11.190620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90cb83b-3d2a-450f-ba20-c5c5d82599ba None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:11.226222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250eea-cdc9-440d-b628-8b5b7ff0f92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:11.226826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250eea-cdc9-440d-b628-8b5b7ff0f92e None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:11.227104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250eea-cdc9-440d-b628-8b5b7ff0f92e None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:11.227375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3250eea-cdc9-440d-b628-8b5b7ff0f92e None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:11.228136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3250eea-cdc9-440d-b628-8b5b7ff0f92e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:11.228436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3250eea-cdc9-440d-b628-8b5b7ff0f92e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:11.229866 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3250eea-cdc9-440d-b628-8b5b7ff0f92e None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:11.247048 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 627/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 10 14:04:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:11.253980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84403656-84a7-4eb4-abe7-1067d6b6373d None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:11.351099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84403656-84a7-4eb4-abe7-1067d6b6373d None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:11.382852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7795e2-ad30-4f50-9acc-d4e05b2ce5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:11.383433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7795e2-ad30-4f50-9acc-d4e05b2ce5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:11.383736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7795e2-ad30-4f50-9acc-d4e05b2ce5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:11.384000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7795e2-ad30-4f50-9acc-d4e05b2ce5b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:11.436017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:11.448863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7795e2-ad30-4f50-9acc-d4e05b2ce5b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:11.450318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7795e2-ad30-4f50-9acc-d4e05b2ce5b6 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:11.544539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 628/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:04:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:04:11.551369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250eea-cdc9-440d-b628-8b5b7ff0f92e None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:11.648962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3250eea-cdc9-440d-b628-8b5b7ff0f92e None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:11.661713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:11.683211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f742944-15a3-4355-8094-da5129b137bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:11.683861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f742944-15a3-4355-8094-da5129b137bd None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:11.684134 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f742944-15a3-4355-8094-da5129b137bd None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:11.684402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f742944-15a3-4355-8094-da5129b137bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:11.694953 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f742944-15a3-4355-8094-da5129b137bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:11.696361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f742944-15a3-4355-8094-da5129b137bd None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-5f742944-15a3-4355-8094-da5129b137bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:04:11.710534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:11.711673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:04:11.712784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:04:11.712784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:11.712784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:11.712784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 628/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:11.717396 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7795e2-ad30-4f50-9acc-d4e05b2ce5b6 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:11.804533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:11.818307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7795e2-ad30-4f50-9acc-d4e05b2ce5b6 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:11.852886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60810e6-21bf-412e-a5e0-1911bc2bc196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:11.853473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60810e6-21bf-412e-a5e0-1911bc2bc196 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:11.853749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60810e6-21bf-412e-a5e0-1911bc2bc196 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:11.854016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60810e6-21bf-412e-a5e0-1911bc2bc196 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:11.854645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60810e6-21bf-412e-a5e0-1911bc2bc196 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:11.854941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60810e6-21bf-412e-a5e0-1911bc2bc196 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:11.856338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60810e6-21bf-412e-a5e0-1911bc2bc196 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:11.867716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 629/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:04:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:11.873395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f742944-15a3-4355-8094-da5129b137bd None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:11.970186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f742944-15a3-4355-8094-da5129b137bd None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:12.002519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10ddc6-a2e8-4f68-87e3-58009c101c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:12.003120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10ddc6-a2e8-4f68-87e3-58009c101c93 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:12.003391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10ddc6-a2e8-4f68-87e3-58009c101c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:12.003689 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c10ddc6-a2e8-4f68-87e3-58009c101c93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:12.009998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10ddc6-a2e8-4f68-87e3-58009c101c93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:12.011427 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c10ddc6-a2e8-4f68-87e3-58009c101c93 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-9c10ddc6-a2e8-4f68-87e3-58009c101c93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:04:12.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:12.023410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:12.024540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:04:12.024540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:04:12.024540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:04:12.024540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:04:12.024540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:04:12.024540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:04:12.024540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:04:12.024540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:12.024540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 629/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:04:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:12.029188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60810e6-21bf-412e-a5e0-1911bc2bc196 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:12.125231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60810e6-21bf-412e-a5e0-1911bc2bc196 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:12.156852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67be9ab-b1f4-4912-950a-e85a08703a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:12.157436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67be9ab-b1f4-4912-950a-e85a08703a19 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:12.157713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67be9ab-b1f4-4912-950a-e85a08703a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:12.157982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67be9ab-b1f4-4912-950a-e85a08703a19 None admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:12.158658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67be9ab-b1f4-4912-950a-e85a08703a19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:12.158959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67be9ab-b1f4-4912-950a-e85a08703a19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:12.160396 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67be9ab-b1f4-4912-950a-e85a08703a19 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:12.172244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 630/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:04:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:12.178142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10ddc6-a2e8-4f68-87e3-58009c101c93 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:12.203718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:12.276039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c10ddc6-a2e8-4f68-87e3-58009c101c93 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:12.308689 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b552ddfd-e3f2-4733-bb65-835266a2e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:12.309266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ddfd-e3f2-4733-bb65-835266a2e06b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:12.309537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ddfd-e3f2-4733-bb65-835266a2e06b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:12.309798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552ddfd-e3f2-4733-bb65-835266a2e06b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:12.310499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552ddfd-e3f2-4733-bb65-835266a2e06b None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=2a6a70a10cfa49348cc8b2b42870909d) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:12.310797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552ddfd-e3f2-4733-bb65-835266a2e06b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:12.312805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b552ddfd-e3f2-4733-bb65-835266a2e06b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:12.317944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 630/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:04:12 2022] GET /identity/v3/role_assignments?user.id=2a6a70a10cfa49348cc8b2b42870909d&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:14.465469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac166-116c-4164-9b7c-a85e9f1691ef None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:14.465922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac166-116c-4164-9b7c-a85e9f1691ef None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:14.466010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfac166-116c-4164-9b7c-a85e9f1691ef None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:14.467367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 631/1261] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:04:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:04:14.473284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91fa5-f0b7-483e-a0a4-df2e77419a81 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:14.473705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91fa5-f0b7-483e-a0a4-df2e77419a81 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:14.473823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b91fa5-f0b7-483e-a0a4-df2e77419a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:14.492423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-79b91fa5-f0b7-483e-a0a4-df2e77419a81 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:04:14.493579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-79b91fa5-f0b7-483e-a0a4-df2e77419a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:04:14.558419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 631/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:04:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:04:14.837818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfac166-116c-4164-9b7c-a85e9f1691ef None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:14.849709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:14.937725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfac166-116c-4164-9b7c-a85e9f1691ef None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:14.970086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac085d3-dfe8-4ca3-a885-2a4ac58f6e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:14.970700 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac085d3-dfe8-4ca3-a885-2a4ac58f6e41 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:14.970975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac085d3-dfe8-4ca3-a885-2a4ac58f6e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:14.971246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac085d3-dfe8-4ca3-a885-2a4ac58f6e41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:14.981120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac085d3-dfe8-4ca3-a885-2a4ac58f6e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-7ac085d3-dfe8-4ca3-a885-2a4ac58f6e41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:04:14.986200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:14.987302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:14.988581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 632/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:04:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:14.993378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b91fa5-f0b7-483e-a0a4-df2e77419a81 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:15.011539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:15.091164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b91fa5-f0b7-483e-a0a4-df2e77419a81 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:15.125295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f679c9-725b-4329-b8f3-887b0d23db49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:15.125843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f679c9-725b-4329-b8f3-887b0d23db49 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:15.126117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f679c9-725b-4329-b8f3-887b0d23db49 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:15.126382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f679c9-725b-4329-b8f3-887b0d23db49 None admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:15.127031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f679c9-725b-4329-b8f3-887b0d23db49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:15.127325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f679c9-725b-4329-b8f3-887b0d23db49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:15.128721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f679c9-725b-4329-b8f3-887b0d23db49 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:15.136482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 632/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:04:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:15.142362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac085d3-dfe8-4ca3-a885-2a4ac58f6e41 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:15.206317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:15.238903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac085d3-dfe8-4ca3-a885-2a4ac58f6e41 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:15.271280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38cc42-a555-4d78-81a1-14ad0aa93864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:15.271917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38cc42-a555-4d78-81a1-14ad0aa93864 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:15.272191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38cc42-a555-4d78-81a1-14ad0aa93864 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:15.272457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38cc42-a555-4d78-81a1-14ad0aa93864 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:15.282577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38cc42-a555-4d78-81a1-14ad0aa93864 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:15.284053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38cc42-a555-4d78-81a1-14ad0aa93864 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-8b38cc42-a555-4d78-81a1-14ad0aa93864 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:04:15.298288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:15.299432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:04:15.300753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:04:15.300753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:15.300753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:15.300753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 633/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:15.305050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f679c9-725b-4329-b8f3-887b0d23db49 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:15.383977 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:15.403100 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f679c9-725b-4329-b8f3-887b0d23db49 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:15.436797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c58526-da04-4101-be9c-ccd3c8b3fb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:15.438097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c58526-da04-4101-be9c-ccd3c8b3fb12 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:15.438751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c58526-da04-4101-be9c-ccd3c8b3fb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:15.438856 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c58526-da04-4101-be9c-ccd3c8b3fb12 None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:15.439425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c58526-da04-4101-be9c-ccd3c8b3fb12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:15.439758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c58526-da04-4101-be9c-ccd3c8b3fb12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:15.441118 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c58526-da04-4101-be9c-ccd3c8b3fb12 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:15.452545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 633/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:15.458291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38cc42-a555-4d78-81a1-14ad0aa93864 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:15.553727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38cc42-a555-4d78-81a1-14ad0aa93864 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:15.572790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:15.586455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181094a6-1747-4840-8be7-97754408a4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:15.587068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181094a6-1747-4840-8be7-97754408a4eb None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:15.587344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181094a6-1747-4840-8be7-97754408a4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:15.587646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181094a6-1747-4840-8be7-97754408a4eb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:15.594799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181094a6-1747-4840-8be7-97754408a4eb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:15.596258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181094a6-1747-4840-8be7-97754408a4eb None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-181094a6-1747-4840-8be7-97754408a4eb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:04:15.607216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:15.608373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:15.610302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 634/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:15.614691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c58526-da04-4101-be9c-ccd3c8b3fb12 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:15.710434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c58526-da04-4101-be9c-ccd3c8b3fb12 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:15.743192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0bc3d-dac1-40d9-a824-45a6086a2fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:15.743801 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0bc3d-dac1-40d9-a824-45a6086a2fcb None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:15.744073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0bc3d-dac1-40d9-a824-45a6086a2fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:15.744338 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b0bc3d-dac1-40d9-a824-45a6086a2fcb None admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:15.745048 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0bc3d-dac1-40d9-a824-45a6086a2fcb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:15.745344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0bc3d-dac1-40d9-a824-45a6086a2fcb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:15.746754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b0bc3d-dac1-40d9-a824-45a6086a2fcb None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:15.755548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:15.763728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 634/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 10 14:04:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:15.770284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181094a6-1747-4840-8be7-97754408a4eb None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:15.876336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181094a6-1747-4840-8be7-97754408a4eb None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:15.908680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40f071-39f2-471b-909e-017c9d3e81af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:15.909297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f071-39f2-471b-909e-017c9d3e81af None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:15.909573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f071-39f2-471b-909e-017c9d3e81af None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:15.909841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe40f071-39f2-471b-909e-017c9d3e81af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:15.919910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40f071-39f2-471b-909e-017c9d3e81af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:15.921344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe40f071-39f2-471b-909e-017c9d3e81af None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-fe40f071-39f2-471b-909e-017c9d3e81af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:04:15.935235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:15.936411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:04:15.938044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:04:15.938044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:15.938044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:15.938044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 635/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:15.942774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0bc3d-dac1-40d9-a824-45a6086a2fcb None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:16.040455 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b0bc3d-dac1-40d9-a824-45a6086a2fcb None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:16.073884 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c34ef-f278-432a-ba03-9ce6a9a5db8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:16.074476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c34ef-f278-432a-ba03-9ce6a9a5db8f None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:16.074751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c34ef-f278-432a-ba03-9ce6a9a5db8f None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:16.075022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92c34ef-f278-432a-ba03-9ce6a9a5db8f None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:16.075683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c34ef-f278-432a-ba03-9ce6a9a5db8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:16.075983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c34ef-f278-432a-ba03-9ce6a9a5db8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:16.077342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92c34ef-f278-432a-ba03-9ce6a9a5db8f None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:16.088491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 635/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:16.094361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40f071-39f2-471b-909e-017c9d3e81af None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:16.126048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:16.192880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe40f071-39f2-471b-909e-017c9d3e81af None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:16.224835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cde52a-bc4e-4cc0-8306-165e325759f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:16.225443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cde52a-bc4e-4cc0-8306-165e325759f9 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:16.225718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cde52a-bc4e-4cc0-8306-165e325759f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:16.225987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cde52a-bc4e-4cc0-8306-165e325759f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:16.232294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cde52a-bc4e-4cc0-8306-165e325759f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:16.233721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cde52a-bc4e-4cc0-8306-165e325759f9 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-79cde52a-bc4e-4cc0-8306-165e325759f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:04:16.245350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:16.246412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:16.247543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:04:16.247543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:04:16.247543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:04:16.247543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:04:16.247543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:04:16.247543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:04:16.247543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:04:16.247543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:16.247543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 636/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:04:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:16.252807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c34ef-f278-432a-ba03-9ce6a9a5db8f None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:16.308820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:16.351169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92c34ef-f278-432a-ba03-9ce6a9a5db8f None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:16.383740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65368496-be4f-46d6-b045-f61d7182adcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:16.384333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368496-be4f-46d6-b045-f61d7182adcc None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:16.384605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368496-be4f-46d6-b045-f61d7182adcc None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:16.384878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65368496-be4f-46d6-b045-f61d7182adcc None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:16.385562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368496-be4f-46d6-b045-f61d7182adcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:16.385864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368496-be4f-46d6-b045-f61d7182adcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:16.387286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65368496-be4f-46d6-b045-f61d7182adcc None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:16.399523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 636/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:04:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:16.405488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cde52a-bc4e-4cc0-8306-165e325759f9 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:16.494822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:16.504099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cde52a-bc4e-4cc0-8306-165e325759f9 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:16.536299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21715b13-d92f-4a39-bc8e-8075c1f10c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:16.536941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21715b13-d92f-4a39-bc8e-8075c1f10c89 None admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:16.537217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21715b13-d92f-4a39-bc8e-8075c1f10c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:16.537487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21715b13-d92f-4a39-bc8e-8075c1f10c89 None admin] PATH_INFO: `/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/2a6a70a10cfa49348cc8b2b42870909d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:16.539331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21715b13-d92f-4a39-bc8e-8075c1f10c89 None admin] RBAC: Authorizing `identity:create_grant(project_id=337395b94d7c4530a3fb24594976c6ca, user_id=2a6a70a10cfa49348cc8b2b42870909d, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:16.540790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21715b13-d92f-4a39-bc8e-8075c1f10c89 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:16.552712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 637/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Sat Sep 10 14:04:16 2022] PUT /identity/v3/projects/337395b94d7c4530a3fb24594976c6ca/users/2a6a70a10cfa49348cc8b2b42870909d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:04:18.720407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60112-bd95-40f4-b414-0ad9efd634cd None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:18.720907 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60112-bd95-40f4-b414-0ad9efd634cd None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:18.720907 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a60112-bd95-40f4-b414-0ad9efd634cd None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:18.722190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 637/1274] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:04:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:04:18.727660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce622a-a2b1-4670-bb47-14eceb4275ad None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:18.728077 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce622a-a2b1-4670-bb47-14eceb4275ad None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:18.728199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce622a-a2b1-4670-bb47-14eceb4275ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:18.746697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e5ce622a-a2b1-4670-bb47-14eceb4275ad None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:04:18.747907 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e5ce622a-a2b1-4670-bb47-14eceb4275ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:04:18.813410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 638/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:04:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:04:19.106196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a60112-bd95-40f4-b414-0ad9efd634cd None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:19.205128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a60112-bd95-40f4-b414-0ad9efd634cd None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:19.221757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:19.238805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32dfaa-84bc-4981-bb61-e8c79201f11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:19.239422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32dfaa-84bc-4981-bb61-e8c79201f11c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:19.239728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32dfaa-84bc-4981-bb61-e8c79201f11c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:19.239998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da32dfaa-84bc-4981-bb61-e8c79201f11c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:19.250271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da32dfaa-84bc-4981-bb61-e8c79201f11c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-da32dfaa-84bc-4981-bb61-e8c79201f11c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 10 14:04:19.255020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:19.256140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:19.257304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 638/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Sat Sep 10 14:04:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:19.262172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce622a-a2b1-4670-bb47-14eceb4275ad None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:19.339670 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:19.358284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ce622a-a2b1-4670-bb47-14eceb4275ad None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:19.391677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945a7c1-3791-429c-b6b0-31a940670212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:19.392271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945a7c1-3791-429c-b6b0-31a940670212 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:19.392546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945a7c1-3791-429c-b6b0-31a940670212 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:19.392812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e945a7c1-3791-429c-b6b0-31a940670212 None admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:19.393456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945a7c1-3791-429c-b6b0-31a940670212 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:19.393752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945a7c1-3791-429c-b6b0-31a940670212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:19.395109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e945a7c1-3791-429c-b6b0-31a940670212 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:19.402767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 639/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Sat Sep 10 14:04:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:19.409073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32dfaa-84bc-4981-bb61-e8c79201f11c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:19.506957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da32dfaa-84bc-4981-bb61-e8c79201f11c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:19.539731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdefd65-6de3-40e5-b877-8d847159cbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:19.540345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdefd65-6de3-40e5-b877-8d847159cbaf None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:19.540620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdefd65-6de3-40e5-b877-8d847159cbaf None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:19.540889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdefd65-6de3-40e5-b877-8d847159cbaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:19.550482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdefd65-6de3-40e5-b877-8d847159cbaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:19.551946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdefd65-6de3-40e5-b877-8d847159cbaf None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-2fdefd65-6de3-40e5-b877-8d847159cbaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:04:19.566738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:19.567971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:04:19.569142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:04:19.569142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:19.569142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:19.569142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 639/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:19.573811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945a7c1-3791-429c-b6b0-31a940670212 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:19.671347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e945a7c1-3791-429c-b6b0-31a940670212 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:19.704512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134c23f-322b-4903-8c03-065c3092e1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:19.705098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134c23f-322b-4903-8c03-065c3092e1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:19.705371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134c23f-322b-4903-8c03-065c3092e1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:19.705637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1134c23f-322b-4903-8c03-065c3092e1b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:19.706257 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134c23f-322b-4903-8c03-065c3092e1b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:19.706552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134c23f-322b-4903-8c03-065c3092e1b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:19.707938 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1134c23f-322b-4903-8c03-065c3092e1b7 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:19.719601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 640/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:04:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:19.725445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdefd65-6de3-40e5-b877-8d847159cbaf None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:19.756166 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:19.832786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdefd65-6de3-40e5-b877-8d847159cbaf None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:19.865320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2b249-55dc-4834-afa5-c39b58d43aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:19.865930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2b249-55dc-4834-afa5-c39b58d43aff None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:19.866206 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2b249-55dc-4834-afa5-c39b58d43aff None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:19.866486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a2b249-55dc-4834-afa5-c39b58d43aff None admin] PATH_INFO: `/v3/users/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:19.873541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2b249-55dc-4834-afa5-c39b58d43aff None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:19.875580 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a2b249-55dc-4834-afa5-c39b58d43aff None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-e8a2b249-55dc-4834-afa5-c39b58d43aff None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 10 14:04:19.886369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:19.891556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:19.892784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 640/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:19.894748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134c23f-322b-4903-8c03-065c3092e1b7 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:19.898500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:19.990645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1134c23f-322b-4903-8c03-065c3092e1b7 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:20.024177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d1253-565f-481b-9855-a99e9e0094bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:20.024765 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d1253-565f-481b-9855-a99e9e0094bc None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:20.025037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d1253-565f-481b-9855-a99e9e0094bc None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:20.025303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d1253-565f-481b-9855-a99e9e0094bc None admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:20.026007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d1253-565f-481b-9855-a99e9e0094bc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:20.026303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d1253-565f-481b-9855-a99e9e0094bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:20.027729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d1253-565f-481b-9855-a99e9e0094bc None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:20.043780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 641/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Sat Sep 10 14:04:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:20.050268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2b249-55dc-4834-afa5-c39b58d43aff None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:20.138899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:20.148816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a2b249-55dc-4834-afa5-c39b58d43aff None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:20.181768 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee118e0-d2f3-4346-8228-e2541cb18d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:20.182358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee118e0-d2f3-4346-8228-e2541cb18d08 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:20.182632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee118e0-d2f3-4346-8228-e2541cb18d08 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:20.182898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee118e0-d2f3-4346-8228-e2541cb18d08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:20.247175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee118e0-d2f3-4346-8228-e2541cb18d08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:20.248595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee118e0-d2f3-4346-8228-e2541cb18d08 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:20.332862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:20.343117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 641/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Sat Sep 10 14:04:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 10 14:04:20.349882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d1253-565f-481b-9855-a99e9e0094bc None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:20.409401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:20.449722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d1253-565f-481b-9855-a99e9e0094bc None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:20.483955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba997c0-0e90-4dd8-9a16-15cdddd2f70e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:20.485323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba997c0-0e90-4dd8-9a16-15cdddd2f70e None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:20.485599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba997c0-0e90-4dd8-9a16-15cdddd2f70e None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:20.485867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba997c0-0e90-4dd8-9a16-15cdddd2f70e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:20.495400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba997c0-0e90-4dd8-9a16-15cdddd2f70e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:20.496855 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba997c0-0e90-4dd8-9a16-15cdddd2f70e None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-9ba997c0-0e90-4dd8-9a16-15cdddd2f70e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 10 14:04:20.511481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:20.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 10 14:04:20.513422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 10 14:04:20.513422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 10 14:04:20.513422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:20.513422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 642/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:04:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:20.518791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee118e0-d2f3-4346-8228-e2541cb18d08 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:20.614746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee118e0-d2f3-4346-8228-e2541cb18d08 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:20.647435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1592874-0649-4695-8cf0-0dc9554ece1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:20.648049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592874-0649-4695-8cf0-0dc9554ece1c None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:20.648321 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592874-0649-4695-8cf0-0dc9554ece1c None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:20.648585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1592874-0649-4695-8cf0-0dc9554ece1c None admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:20.649210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1592874-0649-4695-8cf0-0dc9554ece1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:20.649507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1592874-0649-4695-8cf0-0dc9554ece1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:20.650857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1592874-0649-4695-8cf0-0dc9554ece1c None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:20.662456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 642/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Sat Sep 10 14:04:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:20.668021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba997c0-0e90-4dd8-9a16-15cdddd2f70e None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:20.763382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba997c0-0e90-4dd8-9a16-15cdddd2f70e None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:20.776677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:20.797488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f58f4-f490-4797-88bf-78878742665a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:20.798112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f58f4-f490-4797-88bf-78878742665a None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:20.798391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f58f4-f490-4797-88bf-78878742665a None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:20.798661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6f58f4-f490-4797-88bf-78878742665a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:20.805215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f58f4-f490-4797-88bf-78878742665a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:20.806646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6f58f4-f490-4797-88bf-78878742665a None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application [None req-3e6f58f4-f490-4797-88bf-78878742665a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 10 14:04:20.817719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:20.818588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:20.819685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 10 14:04:20.819685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 10 14:04:20.819685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 10 14:04:20.819685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 10 14:04:20.819685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 10 14:04:20.819685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 10 14:04:20.819685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 10 14:04:20.819685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: ERROR keystone.server.flask.application Sep 10 14:04:20.819685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 643/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Sat Sep 10 14:04:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:20.824621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1592874-0649-4695-8cf0-0dc9554ece1c None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:20.881227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:20.931148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1592874-0649-4695-8cf0-0dc9554ece1c None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:20.963222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc480e45-c883-4901-a29c-377ceb1c8958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:20.963839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc480e45-c883-4901-a29c-377ceb1c8958 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:20.964115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc480e45-c883-4901-a29c-377ceb1c8958 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:20.964382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc480e45-c883-4901-a29c-377ceb1c8958 None admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:20.965063 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc480e45-c883-4901-a29c-377ceb1c8958 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:20.965368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc480e45-c883-4901-a29c-377ceb1c8958 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:20.966792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc480e45-c883-4901-a29c-377ceb1c8958 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:20.979086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 643/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Sat Sep 10 14:04:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:20.985830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f58f4-f490-4797-88bf-78878742665a None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:21.083235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6f58f4-f490-4797-88bf-78878742665a None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:21.114893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e46adf5-bc10-4fe1-8d32-8701b8c94b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:21.115498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46adf5-bc10-4fe1-8d32-8701b8c94b90 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:21.115774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46adf5-bc10-4fe1-8d32-8701b8c94b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:21.116040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e46adf5-bc10-4fe1-8d32-8701b8c94b90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:21.116743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46adf5-bc10-4fe1-8d32-8701b8c94b90 None admin] RBAC: Adding query filter params (role.id=5ef89d7e38744a2aa5e3bdf0511d87d6, scope.project.id=337395b94d7c4530a3fb24594976c6ca, user.id=2a6a70a10cfa49348cc8b2b42870909d) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:21.117041 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46adf5-bc10-4fe1-8d32-8701b8c94b90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:21.118445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e46adf5-bc10-4fe1-8d32-8701b8c94b90 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:21.123981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 644/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Sat Sep 10 14:04:20 2022] GET /identity/v3/role_assignments?user.id=2a6a70a10cfa49348cc8b2b42870909d&scope.project.id=337395b94d7c4530a3fb24594976c6ca&role.id=5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:04:23.286124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b313e0-fc5e-4116-88bd-cfd110110133 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:23.286703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b313e0-fc5e-4116-88bd-cfd110110133 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:23.286703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b313e0-fc5e-4116-88bd-cfd110110133 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:23.288075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 644/1288] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:04:23.293159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57583b00-1346-4db6-8d2e-a3d3f1efe427 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:23.293586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57583b00-1346-4db6-8d2e-a3d3f1efe427 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:23.293705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57583b00-1346-4db6-8d2e-a3d3f1efe427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:23.312264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-57583b00-1346-4db6-8d2e-a3d3f1efe427 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:04:23.313446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-57583b00-1346-4db6-8d2e-a3d3f1efe427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:04:23.341790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:23.378868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 645/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:04:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:04:23.650500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b313e0-fc5e-4116-88bd-cfd110110133 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:23.751122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b313e0-fc5e-4116-88bd-cfd110110133 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:23.761295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:23.784666 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318bc32-667d-424d-8ac3-866bf58a97a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:23.785270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318bc32-667d-424d-8ac3-866bf58a97a8 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:23.785541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318bc32-667d-424d-8ac3-866bf58a97a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:23.785806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318bc32-667d-424d-8ac3-866bf58a97a8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:23.786374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318bc32-667d-424d-8ac3-866bf58a97a8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:23.787756 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318bc32-667d-424d-8ac3-866bf58a97a8 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-3318bc32-667d-424d-8ac3-866bf58a97a8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:23.798276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:23.800065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:23.801365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 645/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 10 14:04:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:23.806263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57583b00-1346-4db6-8d2e-a3d3f1efe427 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:23.915688 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57583b00-1346-4db6-8d2e-a3d3f1efe427 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:23.947958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fac32-6619-47cc-8cad-5e1fc714effc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:23.948557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fac32-6619-47cc-8cad-5e1fc714effc None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:23.948833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fac32-6619-47cc-8cad-5e1fc714effc None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:23.949101 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fac32-6619-47cc-8cad-5e1fc714effc None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:23.949717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fac32-6619-47cc-8cad-5e1fc714effc None admin] RBAC: Adding query filter params (name=placement) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:23.950014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fac32-6619-47cc-8cad-5e1fc714effc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:23.951372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30fac32-6619-47cc-8cad-5e1fc714effc None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:23.956157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 646/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Sep 10 14:04:23 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:23.961953 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318bc32-667d-424d-8ac3-866bf58a97a8 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:24.059557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318bc32-667d-424d-8ac3-866bf58a97a8 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:24.092226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943532d-50bd-46f0-96bb-e8554090a8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:24.092809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943532d-50bd-46f0-96bb-e8554090a8a0 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:24.093081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943532d-50bd-46f0-96bb-e8554090a8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:24.093389 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943532d-50bd-46f0-96bb-e8554090a8a0 None admin] PATH_INFO: `/v3/services` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:24.094004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943532d-50bd-46f0-96bb-e8554090a8a0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:24.094298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943532d-50bd-46f0-96bb-e8554090a8a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:24.095687 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943532d-50bd-46f0-96bb-e8554090a8a0 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:24.100255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 646/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Sep 10 14:04:23 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:26.227423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21131fc7-d5fe-4e6c-952c-baddaaf9d82c None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:26.228012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21131fc7-d5fe-4e6c-952c-baddaaf9d82c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:26.228012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21131fc7-d5fe-4e6c-952c-baddaaf9d82c None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:26.229341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 647/1293] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:04:26.235110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bad32-1d31-4426-be10-cb2d83878d7f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:26.235510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bad32-1d31-4426-be10-cb2d83878d7f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:26.235686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474bad32-1d31-4426-be10-cb2d83878d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:26.254150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-474bad32-1d31-4426-be10-cb2d83878d7f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:04:26.255319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-474bad32-1d31-4426-be10-cb2d83878d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:04:26.284173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:26.320648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 647/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:04:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:04:26.591528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21131fc7-d5fe-4e6c-952c-baddaaf9d82c None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:26.622220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:26.690391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21131fc7-d5fe-4e6c-952c-baddaaf9d82c None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:26.723555 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a31650-ffa5-421e-b6b8-d92aa1ec4621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:26.724147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a31650-ffa5-421e-b6b8-d92aa1ec4621 None admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:26.724425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a31650-ffa5-421e-b6b8-d92aa1ec4621 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:26.724709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a31650-ffa5-421e-b6b8-d92aa1ec4621 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:26.725325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a31650-ffa5-421e-b6b8-d92aa1ec4621 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:26.726677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a31650-ffa5-421e-b6b8-d92aa1ec4621 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:26.740375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 648/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Sat Sep 10 14:04:26 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:04:28.892927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683aa3d-0fd9-49d5-b6e8-845f771e319c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:28.893434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683aa3d-0fd9-49d5-b6e8-845f771e319c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:28.893434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683aa3d-0fd9-49d5-b6e8-845f771e319c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:28.894711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 648/1296] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:04:28.900019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e076d-016f-487a-b84b-4a6edce5c226 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:28.900430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e076d-016f-487a-b84b-4a6edce5c226 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:28.900562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e076d-016f-487a-b84b-4a6edce5c226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:28.919092 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c71e076d-016f-487a-b84b-4a6edce5c226 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:04:28.920299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c71e076d-016f-487a-b84b-4a6edce5c226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:04:28.990752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 649/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:04:28 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:04:29.258846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683aa3d-0fd9-49d5-b6e8-845f771e319c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:29.359430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3683aa3d-0fd9-49d5-b6e8-845f771e319c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:29.393144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555047c-89bc-444e-a4d5-71451830d70b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:29.393767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555047c-89bc-444e-a4d5-71451830d70b None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:29.394047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555047c-89bc-444e-a4d5-71451830d70b None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:29.394319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d555047c-89bc-444e-a4d5-71451830d70b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:29.394903 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555047c-89bc-444e-a4d5-71451830d70b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:29.396308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d555047c-89bc-444e-a4d5-71451830d70b None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d555047c-89bc-444e-a4d5-71451830d70b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:29.405010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:29.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:29.407044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 649/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 10 14:04:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:29.412081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e076d-016f-487a-b84b-4a6edce5c226 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:29.430315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:29.510179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e076d-016f-487a-b84b-4a6edce5c226 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:29.543921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762823e-6a69-4737-8083-2d8250653941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:29.544522 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762823e-6a69-4737-8083-2d8250653941 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:29.544798 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762823e-6a69-4737-8083-2d8250653941 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:29.545070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5762823e-6a69-4737-8083-2d8250653941 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:29.545692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762823e-6a69-4737-8083-2d8250653941 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:29.545995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762823e-6a69-4737-8083-2d8250653941 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:29.547395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5762823e-6a69-4737-8083-2d8250653941 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:29.552410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 650/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Sep 10 14:04:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:29.558379 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555047c-89bc-444e-a4d5-71451830d70b None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:29.589394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:29.657202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d555047c-89bc-444e-a4d5-71451830d70b None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:29.688936 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae8389-8b4b-4c10-8c17-6f8699bc2137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:29.689521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8389-8b4b-4c10-8c17-6f8699bc2137 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:29.689795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8389-8b4b-4c10-8c17-6f8699bc2137 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:29.690066 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae8389-8b4b-4c10-8c17-6f8699bc2137 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:29.690749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae8389-8b4b-4c10-8c17-6f8699bc2137 None admin] RBAC: Adding query filter params (interface=public, service_id=9abd84d93ae745d1b214b55167077464, region_id=RegionOne) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:29.691049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae8389-8b4b-4c10-8c17-6f8699bc2137 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:29.692439 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffae8389-8b4b-4c10-8c17-6f8699bc2137 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:29.697225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 650/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Sat Sep 10 14:04:29 2022] GET /identity/v3/endpoints?service_id=9abd84d93ae745d1b214b55167077464&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:29.703748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762823e-6a69-4737-8083-2d8250653941 None admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:29.784402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:29.804806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5762823e-6a69-4737-8083-2d8250653941 None admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:29.843926 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f69c5-25c6-4da2-b71e-c5a66c2617ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:29.844661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f69c5-25c6-4da2-b71e-c5a66c2617ca None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:29.844936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f69c5-25c6-4da2-b71e-c5a66c2617ca None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:29.845205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f69c5-25c6-4da2-b71e-c5a66c2617ca None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:29.845800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f69c5-25c6-4da2-b71e-c5a66c2617ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:29.847156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f69c5-25c6-4da2-b71e-c5a66c2617ca None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:29.854055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 651/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Sat Sep 10 14:04:29 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:04:32.000596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49e06-6074-460a-83dd-2c8103ae2147 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:32.001178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49e06-6074-460a-83dd-2c8103ae2147 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:32.001178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba49e06-6074-460a-83dd-2c8103ae2147 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:32.002467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 651/1302] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:04:32.007724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b95c98-102f-448c-a0e8-fcf6dcba863a None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:32.008133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b95c98-102f-448c-a0e8-fcf6dcba863a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:32.008274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b95c98-102f-448c-a0e8-fcf6dcba863a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:32.026608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c4b95c98-102f-448c-a0e8-fcf6dcba863a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:04:32.027823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c4b95c98-102f-448c-a0e8-fcf6dcba863a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:04:32.094533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 652/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:04:32 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 10 14:04:32.366889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba49e06-6074-460a-83dd-2c8103ae2147 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:32.460582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:32.467322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba49e06-6074-460a-83dd-2c8103ae2147 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:32.501692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c769d-745d-4276-9a6a-b30dd0bca924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:32.502309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c769d-745d-4276-9a6a-b30dd0bca924 None admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:32.502586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c769d-745d-4276-9a6a-b30dd0bca924 None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:32.502855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c769d-745d-4276-9a6a-b30dd0bca924 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:32.503432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c769d-745d-4276-9a6a-b30dd0bca924 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:32.504822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c769d-745d-4276-9a6a-b30dd0bca924 None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application [None req-d56c769d-745d-4276-9a6a-b30dd0bca924 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 10 14:04:32.513428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 10 14:04:32.514684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.get_or_create( Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application with Lock( Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self._enter() Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application return self.creator() Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application created_value = creator( Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: ERROR keystone.server.flask.application Sep 10 14:04:32.515698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 652/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Sat Sep 10 14:04:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 10 14:04:32.520550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b95c98-102f-448c-a0e8-fcf6dcba863a None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:32.592751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:32.615124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b95c98-102f-448c-a0e8-fcf6dcba863a None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:32.647301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab7765-559c-4bfe-8e87-d184b78b17e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:32.647868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab7765-559c-4bfe-8e87-d184b78b17e1 None admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:32.648143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab7765-559c-4bfe-8e87-d184b78b17e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:32.648410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab7765-559c-4bfe-8e87-d184b78b17e1 None admin] PATH_INFO: `/v3/services` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:32.649026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab7765-559c-4bfe-8e87-d184b78b17e1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:04:32.649321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab7765-559c-4bfe-8e87-d184b78b17e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:32.650671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab7765-559c-4bfe-8e87-d184b78b17e1 None admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:32.655647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 653/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Sat Sep 10 14:04:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:04:32.661381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c769d-745d-4276-9a6a-b30dd0bca924 None admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:04:32.759635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c769d-745d-4276-9a6a-b30dd0bca924 None admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:04:32.792239 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ab18c1-39a5-45e2-877f-6971ac372efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:04:32.792829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab18c1-39a5-45e2-877f-6971ac372efa None admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:04:32.793108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab18c1-39a5-45e2-877f-6971ac372efa None admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:04:32.793374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ab18c1-39a5-45e2-877f-6971ac372efa None admin] PATH_INFO: `/v3/endpoints` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:04:32.794041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab18c1-39a5-45e2-877f-6971ac372efa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:04:32.795405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ab18c1-39a5-45e2-877f-6971ac372efa None admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:04:32.817479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 653/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:04:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:04:42.652692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=30 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:04:42.812660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:36.902871 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebc05e-d2de-450a-8baa-766bc85978a0 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:36.903378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebc05e-d2de-450a-8baa-766bc85978a0 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:36.903673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ebc05e-d2de-450a-8baa-766bc85978a0 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:36.910839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bea834-f385-47fc-be0a-877f6add691f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:36.911895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bea834-f385-47fc-be0a-877f6add691f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:36.912326 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bea834-f385-47fc-be0a-877f6add691f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:36.917362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 654/1307] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:05:36 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:05:37.061503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f0bea834-f385-47fc-be0a-877f6add691f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:05:37.063304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f0bea834-f385-47fc-be0a-877f6add691f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:37.176094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 654/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:05:37.473960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da037e-3d13-4c56-b23e-35561fd25f9e None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:37.474586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da037e-3d13-4c56-b23e-35561fd25f9e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:37.474930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da037e-3d13-4c56-b23e-35561fd25f9e None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:37.476618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 655/1309] 10.222.0.26 () {54 vars in 965 bytes} [Sat Sep 10 14:05:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:05:37.483598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9985bd-14c3-49dc-9b5f-4bb2d6ba1599 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:37.483867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9985bd-14c3-49dc-9b5f-4bb2d6ba1599 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:37.484171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9985bd-14c3-49dc-9b5f-4bb2d6ba1599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:37.568074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7e9985bd-14c3-49dc-9b5f-4bb2d6ba1599 None None] MFA Rules not processed for user `12c71caf31354ec193e3e2da231f1e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:05:37.569285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7e9985bd-14c3-49dc-9b5f-4bb2d6ba1599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:37.592450 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:37.657936 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 655/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Sat Sep 10 14:05:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:05:37.666248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da037e-3d13-4c56-b23e-35561fd25f9e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:05:37.668023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-57da037e-3d13-4c56-b23e-35561fd25f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:37.787605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da037e-3d13-4c56-b23e-35561fd25f9e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:05:37.872143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdbd539-cfdb-46d5-8978-620574e9a523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:05:37.872928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbd539-cfdb-46d5-8978-620574e9a523 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:37.873422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbd539-cfdb-46d5-8978-620574e9a523 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:37.873767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdbd539-cfdb-46d5-8978-620574e9a523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:37.875497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9fdbd539-cfdb-46d5-8978-620574e9a523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:37.936695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:37.995214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbd539-cfdb-46d5-8978-620574e9a523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:05:37.996703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdbd539-cfdb-46d5-8978-620574e9a523 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:05:38.016918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 656/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:05:40.483196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa02421-6a29-43f3-b077-f0af577c19f2 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:40.483789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa02421-6a29-43f3-b077-f0af577c19f2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:40.483789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa02421-6a29-43f3-b077-f0af577c19f2 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:40.485118 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 656/1312] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:05:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:05:40.490621 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da845f-4738-4cf7-b35a-a084221ea402 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:40.491034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da845f-4738-4cf7-b35a-a084221ea402 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:40.491313 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73da845f-4738-4cf7-b35a-a084221ea402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:40.572165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-73da845f-4738-4cf7-b35a-a084221ea402 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:05:40.573372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-73da845f-4738-4cf7-b35a-a084221ea402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:40.672403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 657/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:05:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:05:40.956000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa02421-6a29-43f3-b077-f0af577c19f2 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:05:40.957543 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-afa02421-6a29-43f3-b077-f0af577c19f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:41.068733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa02421-6a29-43f3-b077-f0af577c19f2 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:05:41.143007 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:41.145117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2022bdfa-9d54-41ca-840d-28cd8526b1fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:05:41.145735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022bdfa-9d54-41ca-840d-28cd8526b1fc service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:41.146257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022bdfa-9d54-41ca-840d-28cd8526b1fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:41.146841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2022bdfa-9d54-41ca-840d-28cd8526b1fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:41.148687 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2022bdfa-9d54-41ca-840d-28cd8526b1fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:41.258804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022bdfa-9d54-41ca-840d-28cd8526b1fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:05:41.260217 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2022bdfa-9d54-41ca-840d-28cd8526b1fc service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:05:41.278579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 657/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:05:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:05:43.762806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfac513-2a4f-4b55-80d1-37fe2a445387 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:43.763342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfac513-2a4f-4b55-80d1-37fe2a445387 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:43.763342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfac513-2a4f-4b55-80d1-37fe2a445387 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:43.765784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 658/1315] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:05:43.770918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064db297-de5e-4b03-91ad-afbfbb645e9f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:43.771346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064db297-de5e-4b03-91ad-afbfbb645e9f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:43.771485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064db297-de5e-4b03-91ad-afbfbb645e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:43.838616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-064db297-de5e-4b03-91ad-afbfbb645e9f None None] MFA Rules not processed for user `a072f8abd5f2437cbc748eae7f1f4f83`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:05:43.839839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-064db297-de5e-4b03-91ad-afbfbb645e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:43.892988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:43.937043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 658/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:05:43 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:05:44.222218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfac513-2a4f-4b55-80d1-37fe2a445387 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:05:44.224055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-8bfac513-2a4f-4b55-80d1-37fe2a445387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:44.235098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:44.328408 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfac513-2a4f-4b55-80d1-37fe2a445387 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:05:44.402134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738cfb1-48bd-420f-a601-d459e181943b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:05:44.402732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738cfb1-48bd-420f-a601-d459e181943b service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:44.403011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738cfb1-48bd-420f-a601-d459e181943b service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:44.403282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738cfb1-48bd-420f-a601-d459e181943b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:44.405529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9738cfb1-48bd-420f-a601-d459e181943b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:44.468967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:44.515668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738cfb1-48bd-420f-a601-d459e181943b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:05:44.517099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738cfb1-48bd-420f-a601-d459e181943b service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:05:44.535155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 659/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:05:44 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:05:47.213262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8f36f-9b59-4a19-ad8c-9de508aa504a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:47.213800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8f36f-9b59-4a19-ad8c-9de508aa504a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:47.213800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e8f36f-9b59-4a19-ad8c-9de508aa504a None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:47.215139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 659/1318] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:05:47.221274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc1ace-ace8-4c1a-a37e-419c8b73118a None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:47.221721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc1ace-ace8-4c1a-a37e-419c8b73118a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:47.221819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cc1ace-ace8-4c1a-a37e-419c8b73118a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:47.286138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-79cc1ace-ace8-4c1a-a37e-419c8b73118a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:05:47.291838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-79cc1ace-ace8-4c1a-a37e-419c8b73118a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:47.381234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 660/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:05:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:05:47.663758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8f36f-9b59-4a19-ad8c-9de508aa504a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:05:47.665292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-30e8f36f-9b59-4a19-ad8c-9de508aa504a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:47.767792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e8f36f-9b59-4a19-ad8c-9de508aa504a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:05:47.844689 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8186e1c-ac28-43c3-9631-cbb50c4f99b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:05:47.845279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8186e1c-ac28-43c3-9631-cbb50c4f99b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:47.845552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8186e1c-ac28-43c3-9631-cbb50c4f99b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:47.845819 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8186e1c-ac28-43c3-9631-cbb50c4f99b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:47.847393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f8186e1c-ac28-43c3-9631-cbb50c4f99b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:47.854488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:47.954097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8186e1c-ac28-43c3-9631-cbb50c4f99b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:05:47.955548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8186e1c-ac28-43c3-9631-cbb50c4f99b1 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:05:47.973470 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 660/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:05:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:05:50.777497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16285226-cdda-46b7-bbe8-3e337ffa2cbb None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:50.778046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16285226-cdda-46b7-bbe8-3e337ffa2cbb None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:50.778046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16285226-cdda-46b7-bbe8-3e337ffa2cbb None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:50.779543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 661/1321] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:05:50.784794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a126cd-14a3-4ebc-9492-80027177f60d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:50.785226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a126cd-14a3-4ebc-9492-80027177f60d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:50.785334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a126cd-14a3-4ebc-9492-80027177f60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:50.859740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-22a126cd-14a3-4ebc-9492-80027177f60d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:05:50.860984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-22a126cd-14a3-4ebc-9492-80027177f60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:50.923701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:50.965120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 661/1322] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:05:51.249286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16285226-cdda-46b7-bbe8-3e337ffa2cbb None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:05:51.250906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-16285226-cdda-46b7-bbe8-3e337ffa2cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:51.270422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:51.353949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16285226-cdda-46b7-bbe8-3e337ffa2cbb None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:05:51.426765 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79177c-b7f0-4714-88d8-12cc7cb7ff96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:05:51.427377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79177c-b7f0-4714-88d8-12cc7cb7ff96 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:51.427678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79177c-b7f0-4714-88d8-12cc7cb7ff96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:51.427949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79177c-b7f0-4714-88d8-12cc7cb7ff96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:51.429556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3b79177c-b7f0-4714-88d8-12cc7cb7ff96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:51.503382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:51.540664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79177c-b7f0-4714-88d8-12cc7cb7ff96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:05:51.542109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79177c-b7f0-4714-88d8-12cc7cb7ff96 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:05:51.561732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 662/1323] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:05:53.819415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60354e-8295-4f0c-8d60-da31d9e3c04a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:53.819993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60354e-8295-4f0c-8d60-da31d9e3c04a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:53.819993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60354e-8295-4f0c-8d60-da31d9e3c04a None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:53.821315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 662/1324] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:05:53.827621 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e61d-72fa-4f5f-bfa9-86caa406b46d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:53.828139 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e61d-72fa-4f5f-bfa9-86caa406b46d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:53.828259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09e61d-72fa-4f5f-bfa9-86caa406b46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:53.895839 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-6a09e61d-72fa-4f5f-bfa9-86caa406b46d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:05:53.897036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6a09e61d-72fa-4f5f-bfa9-86caa406b46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:54.001628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 663/1325] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:05:54.285931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60354e-8295-4f0c-8d60-da31d9e3c04a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:05:54.287688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9a60354e-8295-4f0c-8d60-da31d9e3c04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:54.408966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60354e-8295-4f0c-8d60-da31d9e3c04a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:05:54.486793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac96f4-79aa-4ff8-8226-040a8257bc1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:05:54.487397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac96f4-79aa-4ff8-8226-040a8257bc1f service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:54.487702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac96f4-79aa-4ff8-8226-040a8257bc1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:54.487975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac96f4-79aa-4ff8-8226-040a8257bc1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:54.489556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ecac96f4-79aa-4ff8-8226-040a8257bc1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:54.507331 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:54.600274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac96f4-79aa-4ff8-8226-040a8257bc1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:05:54.601644 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac96f4-79aa-4ff8-8226-040a8257bc1f service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:05:54.620147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 663/1326] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:05:56.871539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188cc40-ea73-47a0-b97e-2b273ad1dce9 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:56.872010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188cc40-ea73-47a0-b97e-2b273ad1dce9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:56.872133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3188cc40-ea73-47a0-b97e-2b273ad1dce9 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:56.874290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 664/1327] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:05:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:05:56.879646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a72ef7-eb98-425e-8a58-d83716f9119c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:56.880116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a72ef7-eb98-425e-8a58-d83716f9119c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:56.880211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a72ef7-eb98-425e-8a58-d83716f9119c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:56.946055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e3a72ef7-eb98-425e-8a58-d83716f9119c None None] MFA Rules not processed for user `a072f8abd5f2437cbc748eae7f1f4f83`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:05:56.947174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e3a72ef7-eb98-425e-8a58-d83716f9119c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:57.011670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:57.043320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 664/1328] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:05:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:05:57.344934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a70c-3fee-41b6-8cf3-4e818f4c3922 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:57.345219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a70c-3fee-41b6-8cf3-4e818f4c3922 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:57.345489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a70c-3fee-41b6-8cf3-4e818f4c3922 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:57.346834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 665/1329] 10.222.0.26 () {54 vars in 965 bytes} [Sat Sep 10 14:05:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:05:57.353239 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8a3ac-ccdd-4ec9-875b-2a4df2a5832f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:57.353861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8a3ac-ccdd-4ec9-875b-2a4df2a5832f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:57.353960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8a3ac-ccdd-4ec9-875b-2a4df2a5832f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:57.430303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9ff8a3ac-ccdd-4ec9-875b-2a4df2a5832f None None] MFA Rules not processed for user `12c71caf31354ec193e3e2da231f1e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:05:57.431612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9ff8a3ac-ccdd-4ec9-875b-2a4df2a5832f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:57.509231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 665/1330] 10.222.0.26 () {58 vars in 1062 bytes} [Sat Sep 10 14:05:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:05:57.518219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324a70c-3fee-41b6-8cf3-4e818f4c3922 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:05:57.519760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0324a70c-3fee-41b6-8cf3-4e818f4c3922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:57.551233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:57.626301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324a70c-3fee-41b6-8cf3-4e818f4c3922 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:05:57.701474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c83d9-b0b6-49b3-bd84-6b3ece307be9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:05:57.702153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c83d9-b0b6-49b3-bd84-6b3ece307be9 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:05:57.702452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c83d9-b0b6-49b3-bd84-6b3ece307be9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:05:57.702716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c83d9-b0b6-49b3-bd84-6b3ece307be9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:05:57.704322 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d11c83d9-b0b6-49b3-bd84-6b3ece307be9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:05:57.782253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:05:57.822937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c83d9-b0b6-49b3-bd84-6b3ece307be9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:05:57.824586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c83d9-b0b6-49b3-bd84-6b3ece307be9 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:05:57.844264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 666/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:05:57 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:00.612049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0edbf-c564-48ed-82d4-db14bf73d264 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:00.612613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0edbf-c564-48ed-82d4-db14bf73d264 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:00.612613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a0edbf-c564-48ed-82d4-db14bf73d264 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:00.613927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 666/1332] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:00.619041 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1bc7c-90c8-4034-9d97-f9a0dfa149a8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:00.619493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1bc7c-90c8-4034-9d97-f9a0dfa149a8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:00.619629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1bc7c-90c8-4034-9d97-f9a0dfa149a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:00.684067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-01f1bc7c-90c8-4034-9d97-f9a0dfa149a8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:00.686064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-01f1bc7c-90c8-4034-9d97-f9a0dfa149a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:00.780852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 667/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:01.069808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0edbf-c564-48ed-82d4-db14bf73d264 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:01.071353 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c4a0edbf-c564-48ed-82d4-db14bf73d264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:01.141436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:01.175343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a0edbf-c564-48ed-82d4-db14bf73d264 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:01.246704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bacd6-6d1e-4ef8-b951-8de506fc856b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:01.247366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bacd6-6d1e-4ef8-b951-8de506fc856b service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:01.247854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bacd6-6d1e-4ef8-b951-8de506fc856b service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:01.248127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bacd6-6d1e-4ef8-b951-8de506fc856b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:01.249691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-de7bacd6-6d1e-4ef8-b951-8de506fc856b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:01.356694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bacd6-6d1e-4ef8-b951-8de506fc856b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:01.358141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7bacd6-6d1e-4ef8-b951-8de506fc856b service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:01.367851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:01.376954 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 667/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:03.566898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306c16-7e80-41c0-be20-7980ca17e394 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:03.567497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306c16-7e80-41c0-be20-7980ca17e394 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:03.567497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0306c16-7e80-41c0-be20-7980ca17e394 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:03.568761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 668/1335] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:03.573883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190c1fb-91ff-48cf-8d9e-80b907c6839f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:03.574302 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190c1fb-91ff-48cf-8d9e-80b907c6839f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:03.574408 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a190c1fb-91ff-48cf-8d9e-80b907c6839f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:03.640207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a190c1fb-91ff-48cf-8d9e-80b907c6839f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:03.641392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a190c1fb-91ff-48cf-8d9e-80b907c6839f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:03.737099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 668/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:04.022085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0306c16-7e80-41c0-be20-7980ca17e394 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:04.023650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a0306c16-7e80-41c0-be20-7980ca17e394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:04.063380 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:04.126929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0306c16-7e80-41c0-be20-7980ca17e394 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:04.198220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a0e9a6-ebd8-417d-9354-c87310ffe238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:04.198821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0e9a6-ebd8-417d-9354-c87310ffe238 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:04.199094 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0e9a6-ebd8-417d-9354-c87310ffe238 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:04.199362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a0e9a6-ebd8-417d-9354-c87310ffe238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:04.201026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-06a0e9a6-ebd8-417d-9354-c87310ffe238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:04.282850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:04.308620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0e9a6-ebd8-417d-9354-c87310ffe238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:04.310625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a0e9a6-ebd8-417d-9354-c87310ffe238 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:04.328719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 669/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:06.821108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15f621-ca1f-4465-91da-b5fb6d7db201 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:06.821646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15f621-ca1f-4465-91da-b5fb6d7db201 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:06.821646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15f621-ca1f-4465-91da-b5fb6d7db201 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:06.822901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 669/1338] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:06.828276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b6f43-5f95-42b5-8a4c-c3a1ac17213f None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:06.828669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b6f43-5f95-42b5-8a4c-c3a1ac17213f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:06.828844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b6f43-5f95-42b5-8a4c-c3a1ac17213f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:06.894524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-cd2b6f43-5f95-42b5-8a4c-c3a1ac17213f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:06.895754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-cd2b6f43-5f95-42b5-8a4c-c3a1ac17213f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:06.995108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 670/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:07.275210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15f621-ca1f-4465-91da-b5fb6d7db201 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:07.276794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-db15f621-ca1f-4465-91da-b5fb6d7db201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:07.353661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:07.381718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15f621-ca1f-4465-91da-b5fb6d7db201 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:07.455809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e857871-c31e-4b09-8732-b3374a02ebf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:07.456415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e857871-c31e-4b09-8732-b3374a02ebf1 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:07.456693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e857871-c31e-4b09-8732-b3374a02ebf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:07.456964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e857871-c31e-4b09-8732-b3374a02ebf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:07.458559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8e857871-c31e-4b09-8732-b3374a02ebf1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:07.565085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e857871-c31e-4b09-8732-b3374a02ebf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:07.566443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e857871-c31e-4b09-8732-b3374a02ebf1 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:07.584096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 670/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:11.710024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43085846-3adc-4e50-86d9-9c6299765766 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:11.710549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43085846-3adc-4e50-86d9-9c6299765766 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:11.710549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43085846-3adc-4e50-86d9-9c6299765766 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:11.711888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 671/1341] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:11.717138 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaf0b5-9bcc-4006-a453-4b691e125a3b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:11.717583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaf0b5-9bcc-4006-a453-4b691e125a3b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:11.717688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efaf0b5-9bcc-4006-a453-4b691e125a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:11.723143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:11.784622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4efaf0b5-9bcc-4006-a453-4b691e125a3b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:11.785885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4efaf0b5-9bcc-4006-a453-4b691e125a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:11.883030 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 671/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:06:12.169015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43085846-3adc-4e50-86d9-9c6299765766 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:12.170543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-43085846-3adc-4e50-86d9-9c6299765766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:12.220144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:12.275516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43085846-3adc-4e50-86d9-9c6299765766 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:12.348256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b101ad0-6389-4bcf-b5b1-5b6492f0ed15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:12.348862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101ad0-6389-4bcf-b5b1-5b6492f0ed15 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:12.349398 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101ad0-6389-4bcf-b5b1-5b6492f0ed15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:12.349663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b101ad0-6389-4bcf-b5b1-5b6492f0ed15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:12.351227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4b101ad0-6389-4bcf-b5b1-5b6492f0ed15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:12.445487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:12.462867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b101ad0-6389-4bcf-b5b1-5b6492f0ed15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:12.464346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b101ad0-6389-4bcf-b5b1-5b6492f0ed15 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:12.485369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 672/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:15.298025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae919-45c5-41b4-adb3-00e27c4c9b41 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:15.298576 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae919-45c5-41b4-adb3-00e27c4c9b41 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:15.298576 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3ae919-45c5-41b4-adb3-00e27c4c9b41 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:15.299952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 672/1344] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:15.305031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066b6c5-bf17-4d71-afe4-b12e19fa48f0 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:15.305423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066b6c5-bf17-4d71-afe4-b12e19fa48f0 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:15.305581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4066b6c5-bf17-4d71-afe4-b12e19fa48f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:15.372805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-4066b6c5-bf17-4d71-afe4-b12e19fa48f0 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:15.374028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4066b6c5-bf17-4d71-afe4-b12e19fa48f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:15.474682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 673/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:15.756588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ae919-45c5-41b4-adb3-00e27c4c9b41 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:15.758117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7b3ae919-45c5-41b4-adb3-00e27c4c9b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:15.854262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:15.874770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3ae919-45c5-41b4-adb3-00e27c4c9b41 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:15.950452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72897f20-03a4-41cf-95ae-02e40d49e201 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:15.951053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72897f20-03a4-41cf-95ae-02e40d49e201 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:15.951331 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72897f20-03a4-41cf-95ae-02e40d49e201 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:15.951635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72897f20-03a4-41cf-95ae-02e40d49e201 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:15.953230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-72897f20-03a4-41cf-95ae-02e40d49e201 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:16.065191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72897f20-03a4-41cf-95ae-02e40d49e201 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:16.066563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72897f20-03a4-41cf-95ae-02e40d49e201 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:16.086318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 673/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:20.242421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf665f81-ebb4-4981-a693-8220c14ba9bd None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:20.242888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf665f81-ebb4-4981-a693-8220c14ba9bd None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:20.243283 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf665f81-ebb4-4981-a693-8220c14ba9bd None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:20.244933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 674/1347] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:20.250952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47d89c-8eaa-45e1-80a0-23b1a1612141 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:20.251487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47d89c-8eaa-45e1-80a0-23b1a1612141 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:20.251609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47d89c-8eaa-45e1-80a0-23b1a1612141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:20.268146 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:20.328601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ff47d89c-8eaa-45e1-80a0-23b1a1612141 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:20.329823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ff47d89c-8eaa-45e1-80a0-23b1a1612141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:20.438319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 674/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:20.730254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf665f81-ebb4-4981-a693-8220c14ba9bd None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:20.731999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bf665f81-ebb4-4981-a693-8220c14ba9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:20.789473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:20.835156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf665f81-ebb4-4981-a693-8220c14ba9bd None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:20.908867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b1645c-a642-4bd6-86f4-f7757b497e2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:20.909470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1645c-a642-4bd6-86f4-f7757b497e2f service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:20.909748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1645c-a642-4bd6-86f4-f7757b497e2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:20.910019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b1645c-a642-4bd6-86f4-f7757b497e2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:20.911639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f8b1645c-a642-4bd6-86f4-f7757b497e2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:21.016784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:21.021102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1645c-a642-4bd6-86f4-f7757b497e2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:21.022544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b1645c-a642-4bd6-86f4-f7757b497e2f service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:21.041098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 675/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:25.979602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbbc4c-7bf5-41f6-ba0e-83027cd68cf9 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:25.980526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbbc4c-7bf5-41f6-ba0e-83027cd68cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:25.981510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbbbc4c-7bf5-41f6-ba0e-83027cd68cf9 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:25.983537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 675/1350] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:06:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:25.988868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6abac6-d920-4cdc-9d39-f3f7871ebcc6 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:25.989290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6abac6-d920-4cdc-9d39-f3f7871ebcc6 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:25.989443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6abac6-d920-4cdc-9d39-f3f7871ebcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:26.144811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-8d6abac6-d920-4cdc-9d39-f3f7871ebcc6 None None] MFA Rules not processed for user `2a6a70a10cfa49348cc8b2b42870909d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:26.146439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-8d6abac6-d920-4cdc-9d39-f3f7871ebcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:26.279804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 676/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:06:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:30.374215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe18ec-3d72-4667-9452-a15fa5848a9d None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:30.374760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe18ec-3d72-4667-9452-a15fa5848a9d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:30.374760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efe18ec-3d72-4667-9452-a15fa5848a9d None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:30.376374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 676/1352] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:30.381586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c9daa-1b57-4e44-82e4-fa126fda921c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:30.381997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c9daa-1b57-4e44-82e4-fa126fda921c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:30.382202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c9daa-1b57-4e44-82e4-fa126fda921c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:30.437199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=59 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:30.442258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:30.468787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-b82c9daa-1b57-4e44-82e4-fa126fda921c None None] MFA Rules not processed for user `2a6a70a10cfa49348cc8b2b42870909d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:30.469931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b82c9daa-1b57-4e44-82e4-fa126fda921c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:30.584424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 677/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:06:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:32.027021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8f89c-b19b-42e4-9b33-e3a489642b7c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:32.027585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8f89c-b19b-42e4-9b33-e3a489642b7c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:32.027585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8f89c-b19b-42e4-9b33-e3a489642b7c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:32.029260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 677/1354] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:32.034169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a979f5c-9a92-40e7-b00b-3606b64f5af7 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:32.034571 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a979f5c-9a92-40e7-b00b-3606b64f5af7 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:32.034842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a979f5c-9a92-40e7-b00b-3606b64f5af7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:32.111881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-6a979f5c-9a92-40e7-b00b-3606b64f5af7 None None] MFA Rules not processed for user `2a6a70a10cfa49348cc8b2b42870909d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:32.113080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6a979f5c-9a92-40e7-b00b-3606b64f5af7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:32.155731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:32.223210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 678/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:06:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:35.850826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ecd3-8a6d-44ba-b757-528d5ba441da None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:35.851900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ecd3-8a6d-44ba-b757-528d5ba441da None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:35.852410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f5ecd3-8a6d-44ba-b757-528d5ba441da None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:35.857107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 678/1356] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:35.861356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c761e-0fe2-424e-8e6e-b7145d9dc5bc None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:35.864594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c761e-0fe2-424e-8e6e-b7145d9dc5bc None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:35.865148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c761e-0fe2-424e-8e6e-b7145d9dc5bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:35.944461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7d3c761e-0fe2-424e-8e6e-b7145d9dc5bc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:35.945595 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7d3c761e-0fe2-424e-8e6e-b7145d9dc5bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:36.055934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 679/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:36.364176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93041-0a52-418a-af96-c4d44271c619 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:36.365079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93041-0a52-418a-af96-c4d44271c619 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:36.365560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c93041-0a52-418a-af96-c4d44271c619 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:36.367092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 679/1358] 10.222.0.26 () {54 vars in 962 bytes} [Sat Sep 10 14:06:36 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:36.375701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5070007-f6a4-406c-9a51-33ab5d1a718d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:36.376417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5070007-f6a4-406c-9a51-33ab5d1a718d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:36.377213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5070007-f6a4-406c-9a51-33ab5d1a718d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:36.401386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:36.589569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a5070007-f6a4-406c-9a51-33ab5d1a718d None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:36.590819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a5070007-f6a4-406c-9a51-33ab5d1a718d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:36.719762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 680/1359] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Sep 10 14:06:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:36.727190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c93041-0a52-418a-af96-c4d44271c619 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:36.728808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-09c93041-0a52-418a-af96-c4d44271c619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:36.888555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c93041-0a52-418a-af96-c4d44271c619 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:36.967373 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:37.014518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaf49b-12ed-412c-a6e2-8e2a243fa3ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:37.015141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaf49b-12ed-412c-a6e2-8e2a243fa3ef service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:37.015600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaf49b-12ed-412c-a6e2-8e2a243fa3ef service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:37.015874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaf49b-12ed-412c-a6e2-8e2a243fa3ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:37.017573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-86eaf49b-12ed-412c-a6e2-8e2a243fa3ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:37.123218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eaf49b-12ed-412c-a6e2-8e2a243fa3ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:37.124894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86eaf49b-12ed-412c-a6e2-8e2a243fa3ef service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:37.155048 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 680/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:39.550888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0b5c8-001f-45ac-9c1b-95daf5debf11 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:39.551508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0b5c8-001f-45ac-9c1b-95daf5debf11 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:39.551508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0b5c8-001f-45ac-9c1b-95daf5debf11 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:39.552950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 681/1361] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:39.558214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d8020-01e4-42bf-842a-1fe9394e4460 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:39.558718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d8020-01e4-42bf-842a-1fe9394e4460 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:39.559201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d8020-01e4-42bf-842a-1fe9394e4460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:39.638149 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-014d8020-01e4-42bf-842a-1fe9394e4460 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:39.639378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-014d8020-01e4-42bf-842a-1fe9394e4460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:39.678265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:39.743409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 681/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:40.041856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0b5c8-001f-45ac-9c1b-95daf5debf11 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:40.043761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-22b0b5c8-001f-45ac-9c1b-95daf5debf11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:40.126649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:40.232963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0b5c8-001f-45ac-9c1b-95daf5debf11 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:40.352934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79c607c-45f9-4a64-882a-6f4a6244b94a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:40.353461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c607c-45f9-4a64-882a-6f4a6244b94a service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:40.353736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c607c-45f9-4a64-882a-6f4a6244b94a service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:40.354009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79c607c-45f9-4a64-882a-6f4a6244b94a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:40.356174 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a79c607c-45f9-4a64-882a-6f4a6244b94a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:40.379407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:40.455200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c607c-45f9-4a64-882a-6f4a6244b94a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:40.456849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79c607c-45f9-4a64-882a-6f4a6244b94a service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:40.476641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 682/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:42.987431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9e079-75f4-4a54-9d3b-7e2b61c800c3 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:42.988000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9e079-75f4-4a54-9d3b-7e2b61c800c3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:42.988366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e9e079-75f4-4a54-9d3b-7e2b61c800c3 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:42.991602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 682/1364] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:42.995474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fca8c-c00e-4ed0-8dcf-792d0f797db5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:42.995898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fca8c-c00e-4ed0-8dcf-792d0f797db5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:42.996014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2fca8c-c00e-4ed0-8dcf-792d0f797db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:43.065021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2c2fca8c-c00e-4ed0-8dcf-792d0f797db5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:43.066225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2c2fca8c-c00e-4ed0-8dcf-792d0f797db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:43.178329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:43.183821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 683/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:43.472498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d558ca-cf70-4834-ae02-a29731a7a58a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:43.472926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d558ca-cf70-4834-ae02-a29731a7a58a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:43.473303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d558ca-cf70-4834-ae02-a29731a7a58a None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:43.475129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 683/1366] 10.222.0.26 () {54 vars in 962 bytes} [Sat Sep 10 14:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:43.481512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bce94-4ee0-4963-b951-4f46028a7aa8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:43.481995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bce94-4ee0-4963-b951-4f46028a7aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:43.482268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49bce94-4ee0-4963-b951-4f46028a7aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:43.560761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-f49bce94-4ee0-4963-b951-4f46028a7aa8 None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:43.561959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f49bce94-4ee0-4963-b951-4f46028a7aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:43.692261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 684/1367] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Sep 10 14:06:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:43.700838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d558ca-cf70-4834-ae02-a29731a7a58a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:43.702483 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-67d558ca-cf70-4834-ae02-a29731a7a58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:43.876742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d558ca-cf70-4834-ae02-a29731a7a58a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:43.898889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:44.010844 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba0673-d556-4c4f-91ed-69fb374b3261 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:44.011632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba0673-d556-4c4f-91ed-69fb374b3261 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:44.012637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba0673-d556-4c4f-91ed-69fb374b3261 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:44.012925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba0673-d556-4c4f-91ed-69fb374b3261 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:44.014596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-86ba0673-d556-4c4f-91ed-69fb374b3261 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:44.055725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3374ca-1cc7-4406-a781-66fad94ec290 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:44.056312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3374ca-1cc7-4406-a781-66fad94ec290 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:44.056312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3374ca-1cc7-4406-a781-66fad94ec290 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:44.057743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 685/1368] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:06:44.062632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eecec6-e972-4425-aa97-9827957c5043 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:44.062909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eecec6-e972-4425-aa97-9827957c5043 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:44.063180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14eecec6-e972-4425-aa97-9827957c5043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:44.119495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba0673-d556-4c4f-91ed-69fb374b3261 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:44.121123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba0673-d556-4c4f-91ed-69fb374b3261 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:44.130663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:44.139311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-14eecec6-e972-4425-aa97-9827957c5043 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:44.141476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-14eecec6-e972-4425-aa97-9827957c5043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:44.141622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 684/1369] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:44.259669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 686/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:06:44.601277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd6161-e85c-41b3-a526-578de9ad878e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:44.601561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd6161-e85c-41b3-a526-578de9ad878e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:44.601936 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd6161-e85c-41b3-a526-578de9ad878e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:44.603355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 685/1371] 10.222.0.26 () {54 vars in 964 bytes} [Sat Sep 10 14:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:44.609703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf7f4-489f-407d-88d9-48d8c1de3402 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:44.610248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf7f4-489f-407d-88d9-48d8c1de3402 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:44.610755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85bf7f4-489f-407d-88d9-48d8c1de3402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:44.703561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c85bf7f4-489f-407d-88d9-48d8c1de3402 None None] MFA Rules not processed for user `fdc278f50e794e12a27d30fca9d131c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:44.705108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c85bf7f4-489f-407d-88d9-48d8c1de3402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:44.788030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:44.805413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 687/1372] 10.222.0.26 () {58 vars in 1061 bytes} [Sat Sep 10 14:06:44 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:44.807308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd6161-e85c-41b3-a526-578de9ad878e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:44.810902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e6bd6161-e85c-41b3-a526-578de9ad878e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:44.845243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:44.930951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd6161-e85c-41b3-a526-578de9ad878e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:45.012883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ae4a41-e9b4-4044-ae82-856a6d348c33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:45.013696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae4a41-e9b4-4044-ae82-856a6d348c33 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:45.013995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae4a41-e9b4-4044-ae82-856a6d348c33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:45.014266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ae4a41-e9b4-4044-ae82-856a6d348c33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:45.015914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-28ae4a41-e9b4-4044-ae82-856a6d348c33 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:45.099329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:45.139105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae4a41-e9b4-4044-ae82-856a6d348c33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:45.140607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ae4a41-e9b4-4044-ae82-856a6d348c33 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:45.163951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 686/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:46.404812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9701d61-c360-46e2-b32d-931b4a302387 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:46.405426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9701d61-c360-46e2-b32d-931b4a302387 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:46.405426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9701d61-c360-46e2-b32d-931b4a302387 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:46.406703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 688/1374] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:46.412319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110baf54-e9da-4de1-8c01-ccc6ba45ede9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:46.412758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110baf54-e9da-4de1-8c01-ccc6ba45ede9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:46.412965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110baf54-e9da-4de1-8c01-ccc6ba45ede9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:46.491914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-110baf54-e9da-4de1-8c01-ccc6ba45ede9 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:46.493058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-110baf54-e9da-4de1-8c01-ccc6ba45ede9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:46.592626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 687/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:46.896111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9701d61-c360-46e2-b32d-931b4a302387 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:46.898071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c9701d61-c360-46e2-b32d-931b4a302387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:47.053336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9701d61-c360-46e2-b32d-931b4a302387 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:47.129128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:47.185661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485615d1-97ed-4ea5-a24d-10d4830fed46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:47.186178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485615d1-97ed-4ea5-a24d-10d4830fed46 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:47.186457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485615d1-97ed-4ea5-a24d-10d4830fed46 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:47.186731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485615d1-97ed-4ea5-a24d-10d4830fed46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:47.188392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-485615d1-97ed-4ea5-a24d-10d4830fed46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:47.285458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485615d1-97ed-4ea5-a24d-10d4830fed46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:47.287134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485615d1-97ed-4ea5-a24d-10d4830fed46 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:47.306029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 689/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:49.581987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ebcf-3591-434a-a000-77da8572917f None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:49.582581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ebcf-3591-434a-a000-77da8572917f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:49.582581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb1ebcf-3591-434a-a000-77da8572917f None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:49.584076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 688/1377] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:49.589292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15343b2e-3e01-4284-b814-c89a341d9649 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:49.589729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15343b2e-3e01-4284-b814-c89a341d9649 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:49.589867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15343b2e-3e01-4284-b814-c89a341d9649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:49.659550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-15343b2e-3e01-4284-b814-c89a341d9649 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:49.660763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-15343b2e-3e01-4284-b814-c89a341d9649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:49.691554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:49.763750 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 690/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:50.057282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1ebcf-3591-434a-a000-77da8572917f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:50.058804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3cb1ebcf-3591-434a-a000-77da8572917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:50.099485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:50.221118 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb1ebcf-3591-434a-a000-77da8572917f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:50.339087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:50.344974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebb4f1-0348-41b0-a06d-179fb95264ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:50.346071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebb4f1-0348-41b0-a06d-179fb95264ff service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:50.346462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebb4f1-0348-41b0-a06d-179fb95264ff service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:50.348965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebb4f1-0348-41b0-a06d-179fb95264ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:50.349110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-75ebb4f1-0348-41b0-a06d-179fb95264ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:50.455273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebb4f1-0348-41b0-a06d-179fb95264ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:50.457197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebb4f1-0348-41b0-a06d-179fb95264ff service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:50.476951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 689/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:52.704478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c0755-9c62-4fd2-9a82-eceac25a6206 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:52.705154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c0755-9c62-4fd2-9a82-eceac25a6206 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:52.705154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c0755-9c62-4fd2-9a82-eceac25a6206 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:52.710770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 691/1380] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:52.712186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d49c-32b4-4c80-80f3-ffdc08a3bc3b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:52.712746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d49c-32b4-4c80-80f3-ffdc08a3bc3b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:52.713095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56d49c-32b4-4c80-80f3-ffdc08a3bc3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:52.825806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5a56d49c-32b4-4c80-80f3-ffdc08a3bc3b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:52.827016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5a56d49c-32b4-4c80-80f3-ffdc08a3bc3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:52.900140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:52.934282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 690/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:53.228143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c0755-9c62-4fd2-9a82-eceac25a6206 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:53.229646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-761c0755-9c62-4fd2-9a82-eceac25a6206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:53.399358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c0755-9c62-4fd2-9a82-eceac25a6206 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:53.414356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:53.525521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f58fda3-394c-4e44-9cf5-9ad367de6dea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:53.526044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58fda3-394c-4e44-9cf5-9ad367de6dea service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:53.526315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58fda3-394c-4e44-9cf5-9ad367de6dea service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:53.526577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f58fda3-394c-4e44-9cf5-9ad367de6dea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:53.529207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0f58fda3-394c-4e44-9cf5-9ad367de6dea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:53.622574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58fda3-394c-4e44-9cf5-9ad367de6dea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:53.624208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f58fda3-394c-4e44-9cf5-9ad367de6dea service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:53.643515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 692/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:56.041242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46469bbb-4d07-4a4c-9a8b-f9043050f19d None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:56.041768 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46469bbb-4d07-4a4c-9a8b-f9043050f19d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:56.041879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46469bbb-4d07-4a4c-9a8b-f9043050f19d None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:56.043176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 691/1383] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:56.048481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacdf53-426b-4e42-982b-57914c272560 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:56.048916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacdf53-426b-4e42-982b-57914c272560 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:56.049020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cacdf53-426b-4e42-982b-57914c272560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:56.070952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:56.119853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-3cacdf53-426b-4e42-982b-57914c272560 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:56.121480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3cacdf53-426b-4e42-982b-57914c272560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:56.233526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 693/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:56.527009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46469bbb-4d07-4a4c-9a8b-f9043050f19d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:56.529217 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-46469bbb-4d07-4a4c-9a8b-f9043050f19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:56.687934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46469bbb-4d07-4a4c-9a8b-f9043050f19d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:56.752646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:56.830308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a867bc00-4a38-4760-973e-348415660d50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:56.831312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867bc00-4a38-4760-973e-348415660d50 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:56.831979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867bc00-4a38-4760-973e-348415660d50 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:56.832323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867bc00-4a38-4760-973e-348415660d50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:56.834457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a867bc00-4a38-4760-973e-348415660d50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:56.936436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867bc00-4a38-4760-973e-348415660d50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:06:56.938052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a867bc00-4a38-4760-973e-348415660d50 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:06:56.957362 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 692/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:06:59.206797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d424ab-a7f5-4839-a675-251f0b8ff3cd None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:59.207369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d424ab-a7f5-4839-a675-251f0b8ff3cd None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:59.207369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d424ab-a7f5-4839-a675-251f0b8ff3cd None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:59.208725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 694/1386] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:06:59.214125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f579289-ba89-4ad4-b7b8-fa3878239af6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:59.214550 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f579289-ba89-4ad4-b7b8-fa3878239af6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:59.215245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f579289-ba89-4ad4-b7b8-fa3878239af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:59.285796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3f579289-ba89-4ad4-b7b8-fa3878239af6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:06:59.287005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3f579289-ba89-4ad4-b7b8-fa3878239af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:59.295799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:59.384379 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 693/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:06:59.677797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d424ab-a7f5-4839-a675-251f0b8ff3cd None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:06:59.679295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-53d424ab-a7f5-4839-a675-251f0b8ff3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:06:59.782355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:06:59.836676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d424ab-a7f5-4839-a675-251f0b8ff3cd None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:06:59.950073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2e780d-4964-4451-b055-4dcee0dcc3a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:06:59.950676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e780d-4964-4451-b055-4dcee0dcc3a1 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:06:59.950949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e780d-4964-4451-b055-4dcee0dcc3a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:06:59.951213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2e780d-4964-4451-b055-4dcee0dcc3a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:06:59.952815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-cb2e780d-4964-4451-b055-4dcee0dcc3a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:00.013823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:00.045478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e780d-4964-4451-b055-4dcee0dcc3a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:00.047074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2e780d-4964-4451-b055-4dcee0dcc3a1 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:00.065286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 695/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:00.550072 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb44c-ec58-4928-929b-2a21e3c741f4 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:00.550641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb44c-ec58-4928-929b-2a21e3c741f4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:00.550641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb44c-ec58-4928-929b-2a21e3c741f4 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:00.551928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 694/1389] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:00.557477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631507aa-befe-495d-9770-2e0ad57ab554 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:00.557933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631507aa-befe-495d-9770-2e0ad57ab554 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:00.558036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631507aa-befe-495d-9770-2e0ad57ab554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:00.626513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-631507aa-befe-495d-9770-2e0ad57ab554 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:00.627756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-631507aa-befe-495d-9770-2e0ad57ab554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:00.725894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 696/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:01.027872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613f962-c1dc-4185-93d5-a4d5a1af4967 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:01.028158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613f962-c1dc-4185-93d5-a4d5a1af4967 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:01.028453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613f962-c1dc-4185-93d5-a4d5a1af4967 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:01.029813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 695/1391] 10.222.0.26 () {54 vars in 966 bytes} [Sat Sep 10 14:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:01.035168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a8e77-b463-494d-9ef1-6fcc0d2b70b0 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:01.035482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a8e77-b463-494d-9ef1-6fcc0d2b70b0 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:01.035757 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a8e77-b463-494d-9ef1-6fcc0d2b70b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:01.081492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:01.113919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d34a8e77-b463-494d-9ef1-6fcc0d2b70b0 None None] MFA Rules not processed for user `cf75b583add44b6b87bff5dc67080d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:01.115121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d34a8e77-b463-494d-9ef1-6fcc0d2b70b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:01.190316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 697/1392] 10.222.0.26 () {58 vars in 1063 bytes} [Sat Sep 10 14:07:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:01.200026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613f962-c1dc-4185-93d5-a4d5a1af4967 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:01.201515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6613f962-c1dc-4185-93d5-a4d5a1af4967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:01.307738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613f962-c1dc-4185-93d5-a4d5a1af4967 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:01.348318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:01.385174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a19c4b-ec1f-44e8-9550-7ed25c4dcd4e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf75b583add44b6b87bff5dc67080d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:01.385892 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19c4b-ec1f-44e8-9550-7ed25c4dcd4e service glance] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:01.386174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19c4b-ec1f-44e8-9550-7ed25c4dcd4e service glance] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:01.386455 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a19c4b-ec1f-44e8-9550-7ed25c4dcd4e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:01.388622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-23a19c4b-ec1f-44e8-9550-7ed25c4dcd4e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:01.501351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a19c4b-ec1f-44e8-9550-7ed25c4dcd4e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:01.502810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a19c4b-ec1f-44e8-9550-7ed25c4dcd4e service glance] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:01.523099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 696/1393] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:02.284545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ce605-5938-4b20-9598-ef612df63f26 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:02.285083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ce605-5938-4b20-9598-ef612df63f26 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:02.285190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50ce605-5938-4b20-9598-ef612df63f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:02.287221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0952b4-9d57-42a9-8f69-1b50abf8ce54 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:02.287660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0952b4-9d57-42a9-8f69-1b50abf8ce54 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:02.287806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0952b4-9d57-42a9-8f69-1b50abf8ce54 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:02.289130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 697/1394] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:07:02.294642 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63db68-f074-4440-b957-6da56a03a738 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:02.294917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63db68-f074-4440-b957-6da56a03a738 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:02.295186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f63db68-f074-4440-b957-6da56a03a738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:02.355080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-f50ce605-5938-4b20-9598-ef612df63f26 None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:02.356386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f50ce605-5938-4b20-9598-ef612df63f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:02.362098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6f63db68-f074-4440-b957-6da56a03a738 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:02.363283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6f63db68-f074-4440-b957-6da56a03a738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:02.381714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:02.441470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:02.449923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 698/1395] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:07:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:02.462947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 698/1396] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:07:02.497813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fab370-c057-4b4a-9b6c-4c2831e25d59 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:02.498092 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fab370-c057-4b4a-9b6c-4c2831e25d59 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:02.498359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fab370-c057-4b4a-9b6c-4c2831e25d59 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:02.499776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 699/1397] 10.222.0.26 () {54 vars in 943 bytes} [Sat Sep 10 14:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:02.506209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a0571-3ba4-414a-8b56-4cf5c8bacf62 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:02.506494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a0571-3ba4-414a-8b56-4cf5c8bacf62 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:02.506765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730a0571-3ba4-414a-8b56-4cf5c8bacf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:02.585382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-730a0571-3ba4-414a-8b56-4cf5c8bacf62 None None] MFA Rules not processed for user `bc0a7e1b772b4703ac2e5ec33b5ecf6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:02.586583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-730a0571-3ba4-414a-8b56-4cf5c8bacf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:02.668946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:02.695705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 699/1398] 10.222.0.26 () {58 vars in 1040 bytes} [Sat Sep 10 14:07:02 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:02.705267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fab370-c057-4b4a-9b6c-4c2831e25d59 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:02.706769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-66fab370-c057-4b4a-9b6c-4c2831e25d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:02.755277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a0571-3ba4-414a-8b56-4cf5c8bacf62 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:02.756507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-730a0571-3ba4-414a-8b56-4cf5c8bacf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:02.850893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fab370-c057-4b4a-9b6c-4c2831e25d59 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:02.860354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730a0571-3ba4-414a-8b56-4cf5c8bacf62 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:02.929232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:02.956052 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86c068-b427-40c9-8a3c-861839136787 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:02.956669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86c068-b427-40c9-8a3c-861839136787 service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:02.956947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86c068-b427-40c9-8a3c-861839136787 service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:02.957219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86c068-b427-40c9-8a3c-861839136787 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:02.958869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6b86c068-b427-40c9-8a3c-861839136787 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:02.974682 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:02.986139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b320c03-e64a-41b7-a37c-4159dc6eb320 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:02.986748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b320c03-e64a-41b7-a37c-4159dc6eb320 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:02.987018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b320c03-e64a-41b7-a37c-4159dc6eb320 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:02.987283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b320c03-e64a-41b7-a37c-4159dc6eb320 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:02.988898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0b320c03-e64a-41b7-a37c-4159dc6eb320 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:03.045855 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86c068-b427-40c9-8a3c-861839136787 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:03.047507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86c068-b427-40c9-8a3c-861839136787 service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:03.059612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 700/1399] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:07:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:07:03.079928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b320c03-e64a-41b7-a37c-4159dc6eb320 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:03.081576 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b320c03-e64a-41b7-a37c-4159dc6eb320 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:03.102051 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 700/1400] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:05.393674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa1ccb-cdae-4791-8c46-859eeaa906bc None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:05.394237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa1ccb-cdae-4791-8c46-859eeaa906bc None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:05.394237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa1ccb-cdae-4791-8c46-859eeaa906bc None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:05.399587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 701/1401] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:07:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:05.401327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414903e2-4bc2-453b-8619-c45d14683317 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:05.401790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414903e2-4bc2-453b-8619-c45d14683317 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:05.401886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414903e2-4bc2-453b-8619-c45d14683317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:05.483017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-414903e2-4bc2-453b-8619-c45d14683317 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:05.484258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-414903e2-4bc2-453b-8619-c45d14683317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:05.547711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:05.591264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 701/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:05.886689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa1ccb-cdae-4791-8c46-859eeaa906bc None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:05.888219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-18fa1ccb-cdae-4791-8c46-859eeaa906bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:06.015905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:06.055035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa1ccb-cdae-4791-8c46-859eeaa906bc None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:06.195735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1dc00c-688f-4aff-b11d-b87b61fc3867 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:06.196346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dc00c-688f-4aff-b11d-b87b61fc3867 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:06.196626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dc00c-688f-4aff-b11d-b87b61fc3867 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:06.196896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dc00c-688f-4aff-b11d-b87b61fc3867 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:06.198519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6e1dc00c-688f-4aff-b11d-b87b61fc3867 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:06.291482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:06.305821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dc00c-688f-4aff-b11d-b87b61fc3867 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:06.307345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dc00c-688f-4aff-b11d-b87b61fc3867 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:06.328429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 702/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:08.611835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20b860-60cc-411b-b793-da42a8bb6a51 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:08.612414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20b860-60cc-411b-b793-da42a8bb6a51 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:08.612414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf20b860-60cc-411b-b793-da42a8bb6a51 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:08.613704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 702/1404] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:08.618911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f580-7b1f-4e60-a749-17f9862756e1 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:08.619325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f580-7b1f-4e60-a749-17f9862756e1 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:08.619574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d4f580-7b1f-4e60-a749-17f9862756e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:08.690937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c5d4f580-7b1f-4e60-a749-17f9862756e1 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:08.692207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c5d4f580-7b1f-4e60-a749-17f9862756e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:08.799512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 703/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:09.096655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20b860-60cc-411b-b793-da42a8bb6a51 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:09.098228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bf20b860-60cc-411b-b793-da42a8bb6a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:09.266044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf20b860-60cc-411b-b793-da42a8bb6a51 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:09.320056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:09.394051 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ab3b78-1a84-4ca6-a000-9cdcd44f3153 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:09.394567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab3b78-1a84-4ca6-a000-9cdcd44f3153 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:09.394848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab3b78-1a84-4ca6-a000-9cdcd44f3153 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:09.395120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ab3b78-1a84-4ca6-a000-9cdcd44f3153 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:09.396747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-50ab3b78-1a84-4ca6-a000-9cdcd44f3153 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:09.501117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab3b78-1a84-4ca6-a000-9cdcd44f3153 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:09.502729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ab3b78-1a84-4ca6-a000-9cdcd44f3153 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:09.521848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 703/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:11.735612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90206e-df87-45b2-a2a9-d7a203e07a54 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:11.736145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90206e-df87-45b2-a2a9-d7a203e07a54 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:11.736145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90206e-df87-45b2-a2a9-d7a203e07a54 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:11.737462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 704/1407] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:11.742772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f1d7-10cc-47b2-8842-cc36a3f8dc8b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:11.743572 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f1d7-10cc-47b2-8842-cc36a3f8dc8b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:11.743851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652f1d7-10cc-47b2-8842-cc36a3f8dc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:11.827662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a652f1d7-10cc-47b2-8842-cc36a3f8dc8b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:11.828902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a652f1d7-10cc-47b2-8842-cc36a3f8dc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:11.831942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:11.943939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 704/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:12.238316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90206e-df87-45b2-a2a9-d7a203e07a54 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:12.239870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ea90206e-df87-45b2-a2a9-d7a203e07a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:12.308993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:12.408530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90206e-df87-45b2-a2a9-d7a203e07a54 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:12.536873 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4767c22e-e1d2-4fd1-9172-f82c5dd0815d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:12.537503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767c22e-e1d2-4fd1-9172-f82c5dd0815d service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:12.537848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767c22e-e1d2-4fd1-9172-f82c5dd0815d service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:12.538189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4767c22e-e1d2-4fd1-9172-f82c5dd0815d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:12.539930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4767c22e-e1d2-4fd1-9172-f82c5dd0815d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:12.562810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:12.644532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767c22e-e1d2-4fd1-9172-f82c5dd0815d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:12.646243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4767c22e-e1d2-4fd1-9172-f82c5dd0815d service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:12.667314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 705/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:21.942577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=41 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:22.664936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=37 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:25.872410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fffec3-fe57-4347-9840-9b3f2fcc3e37 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:25.872933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fffec3-fe57-4347-9840-9b3f2fcc3e37 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:25.872933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fffec3-fe57-4347-9840-9b3f2fcc3e37 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:25.874237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 705/1410] 10.222.0.26 () {54 vars in 918 bytes} [Sat Sep 10 14:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:25.879478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784336ce-1cca-426d-b503-d466b652fa05 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:25.879932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784336ce-1cca-426d-b503-d466b652fa05 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:25.880036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784336ce-1cca-426d-b503-d466b652fa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:25.955116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-784336ce-1cca-426d-b503-d466b652fa05 None None] MFA Rules not processed for user `2a6a70a10cfa49348cc8b2b42870909d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:25.956351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-784336ce-1cca-426d-b503-d466b652fa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:26.062158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 706/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 10 14:07:25 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:26.076609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6ce50-4aa7-4843-a3bd-66e28a4907f3 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:26.076888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6ce50-4aa7-4843-a3bd-66e28a4907f3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:26.077155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b6ce50-4aa7-4843-a3bd-66e28a4907f3 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:26.078474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 706/1412] 10.222.0.26 () {54 vars in 918 bytes} [Sat Sep 10 14:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:26.083153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2ed65-9455-4574-93cd-d70d051447de None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:26.083432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2ed65-9455-4574-93cd-d70d051447de None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:26.083731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be2ed65-9455-4574-93cd-d70d051447de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:26.157828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:26.160408 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-6be2ed65-9455-4574-93cd-d70d051447de None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:26.162128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6be2ed65-9455-4574-93cd-d70d051447de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:26.278441 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 707/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Sat Sep 10 14:07:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:28.877630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b7d5d-eea7-4794-bfd1-8fb51e1ec4d1 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:28.878201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b7d5d-eea7-4794-bfd1-8fb51e1ec4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:28.878201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b7d5d-eea7-4794-bfd1-8fb51e1ec4d1 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:28.879583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:28.885034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5b89f-e6b8-4bd9-a047-0a739326250c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:28.885466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5b89f-e6b8-4bd9-a047-0a739326250c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:28.885563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5b89f-e6b8-4bd9-a047-0a739326250c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:28.955274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-26b5b89f-e6b8-4bd9-a047-0a739326250c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:28.956497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-26b5b89f-e6b8-4bd9-a047-0a739326250c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:29.014569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:29.056412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:07:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:29.344246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b7d5d-eea7-4794-bfd1-8fb51e1ec4d1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:29.345756 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-843b7d5d-eea7-4794-bfd1-8fb51e1ec4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:29.492869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b7d5d-eea7-4794-bfd1-8fb51e1ec4d1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:29.566822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:29.608998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58170c6-f88c-4e45-b1c7-d3297edb28b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:29.609523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58170c6-f88c-4e45-b1c7-d3297edb28b4 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:29.609795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58170c6-f88c-4e45-b1c7-d3297edb28b4 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:29.610061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58170c6-f88c-4e45-b1c7-d3297edb28b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:29.611666 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d58170c6-f88c-4e45-b1c7-d3297edb28b4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:29.704305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58170c6-f88c-4e45-b1c7-d3297edb28b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:29.705899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58170c6-f88c-4e45-b1c7-d3297edb28b4 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:29.724043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 708/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:30.878892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d55067-43a0-4118-8d03-11efdea8cc7d None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:30.879632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d55067-43a0-4118-8d03-11efdea8cc7d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:30.879632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d55067-43a0-4118-8d03-11efdea8cc7d None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:30.880838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:30.887870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6a971-fe35-404a-bad0-781a3e6c3d7f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:30.888400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6a971-fe35-404a-bad0-781a3e6c3d7f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:30.888505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f6a971-fe35-404a-bad0-781a3e6c3d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:30.963374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-61f6a971-fe35-404a-bad0-781a3e6c3d7f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:30.965176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-61f6a971-fe35-404a-bad0-781a3e6c3d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:30.992555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:31.065553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:07:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:31.217489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d55067-43a0-4118-8d03-11efdea8cc7d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:31.219029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-21d55067-43a0-4118-8d03-11efdea8cc7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:31.364741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d55067-43a0-4118-8d03-11efdea8cc7d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:31.405210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:31.478277 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255d0e7-b03b-4111-9f2c-59da1388d599 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:31.478808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255d0e7-b03b-4111-9f2c-59da1388d599 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:31.479080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255d0e7-b03b-4111-9f2c-59da1388d599 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:31.479346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255d0e7-b03b-4111-9f2c-59da1388d599 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:31.480968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7255d0e7-b03b-4111-9f2c-59da1388d599 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:31.567554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255d0e7-b03b-4111-9f2c-59da1388d599 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:31.569127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7255d0e7-b03b-4111-9f2c-59da1388d599 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:31.588218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 710/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:32.718604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638dc21-3bfc-46fd-8a77-a47e2ba1d3a5 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:32.719191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638dc21-3bfc-46fd-8a77-a47e2ba1d3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:32.719191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638dc21-3bfc-46fd-8a77-a47e2ba1d3a5 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:32.720573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:07:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:32.726878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f6bac-a370-4eca-bdf0-850043c854bb None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:32.727322 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f6bac-a370-4eca-bdf0-850043c854bb None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:32.727421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f6bac-a370-4eca-bdf0-850043c854bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:32.782415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:32.804980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e44f6bac-a370-4eca-bdf0-850043c854bb None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:32.806151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e44f6bac-a370-4eca-bdf0-850043c854bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:32.910071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:07:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:33.062038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638dc21-3bfc-46fd-8a77-a47e2ba1d3a5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:33.063566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0638dc21-3bfc-46fd-8a77-a47e2ba1d3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:33.207094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638dc21-3bfc-46fd-8a77-a47e2ba1d3a5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:33.220068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:33.325537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc139f3-599c-45ae-8c8a-d8ebd11fa9e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:33.326068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc139f3-599c-45ae-8c8a-d8ebd11fa9e2 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:33.326338 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc139f3-599c-45ae-8c8a-d8ebd11fa9e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:33.326604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc139f3-599c-45ae-8c8a-d8ebd11fa9e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:33.328205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4bc139f3-599c-45ae-8c8a-d8ebd11fa9e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:33.417262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc139f3-599c-45ae-8c8a-d8ebd11fa9e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:33.419011 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc139f3-599c-45ae-8c8a-d8ebd11fa9e2 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:33.438219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 711/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:34.592576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18afde-b880-4c30-ae01-e322345ae982 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:34.593142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18afde-b880-4c30-ae01-e322345ae982 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:34.593142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18afde-b880-4c30-ae01-e322345ae982 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:34.594391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:07:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:34.601828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9458ca-d0a8-4463-9338-1caf44f9e98d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:34.602358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9458ca-d0a8-4463-9338-1caf44f9e98d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:34.602358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9458ca-d0a8-4463-9338-1caf44f9e98d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:34.628776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:34.691605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3e9458ca-d0a8-4463-9338-1caf44f9e98d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:34.692835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3e9458ca-d0a8-4463-9338-1caf44f9e98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:34.802456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:07:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:34.977737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18afde-b880-4c30-ae01-e322345ae982 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:34.980610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-af18afde-b880-4c30-ae01-e322345ae982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:35.109431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:35.148242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18afde-b880-4c30-ae01-e322345ae982 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:35.265917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6316287-60cd-4337-91a6-22f6199774a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:35.266436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6316287-60cd-4337-91a6-22f6199774a6 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:35.266709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6316287-60cd-4337-91a6-22f6199774a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:35.267003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6316287-60cd-4337-91a6-22f6199774a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:35.268629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a6316287-60cd-4337-91a6-22f6199774a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:35.344044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:35.359276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6316287-60cd-4337-91a6-22f6199774a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:35.361002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6316287-60cd-4337-91a6-22f6199774a6 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:35.378848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 713/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:36.514173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b2e71-8778-4181-b877-c12e6cbbbe9d None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:36.514673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b2e71-8778-4181-b877-c12e6cbbbe9d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:36.514673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b2e71-8778-4181-b877-c12e6cbbbe9d None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:36.516026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:36.522250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde7a13-ef36-4715-b998-0e7c2a6fd117 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:36.522705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde7a13-ef36-4715-b998-0e7c2a6fd117 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:36.522824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bde7a13-ef36-4715-b998-0e7c2a6fd117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:36.598817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-9bde7a13-ef36-4715-b998-0e7c2a6fd117 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:36.600054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9bde7a13-ef36-4715-b998-0e7c2a6fd117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:36.697424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:07:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:36.851332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b2e71-8778-4181-b877-c12e6cbbbe9d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:36.852878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c62b2e71-8778-4181-b877-c12e6cbbbe9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:36.937397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:36.999150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b2e71-8778-4181-b877-c12e6cbbbe9d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:37.114903 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457d01e-ee13-4550-98e2-79ed0fb7ff5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:37.115426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d01e-ee13-4550-98e2-79ed0fb7ff5c service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:37.115728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d01e-ee13-4550-98e2-79ed0fb7ff5c service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:37.115992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457d01e-ee13-4550-98e2-79ed0fb7ff5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:37.117567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5457d01e-ee13-4550-98e2-79ed0fb7ff5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:37.166413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:37.207661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457d01e-ee13-4550-98e2-79ed0fb7ff5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:37.209263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457d01e-ee13-4550-98e2-79ed0fb7ff5c service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:37.227615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 714/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:38.288546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80946d8-02f3-476a-9f3e-d22609b995cb None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:38.289027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80946d8-02f3-476a-9f3e-d22609b995cb None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:38.289133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80946d8-02f3-476a-9f3e-d22609b995cb None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:38.290453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:38.297193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be8b69-74fc-474a-9cfe-fe52d111bb01 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:38.297640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be8b69-74fc-474a-9cfe-fe52d111bb01 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:38.297750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be8b69-74fc-474a-9cfe-fe52d111bb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:38.373715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-42be8b69-74fc-474a-9cfe-fe52d111bb01 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:38.374908 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-42be8b69-74fc-474a-9cfe-fe52d111bb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:38.471344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:07:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:38.624179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80946d8-02f3-476a-9f3e-d22609b995cb None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:38.625748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e80946d8-02f3-476a-9f3e-d22609b995cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:38.674501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:38.771297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80946d8-02f3-476a-9f3e-d22609b995cb None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:38.884787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c113ab14-a37f-4319-837b-0353c6bc36c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:38.885390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113ab14-a37f-4319-837b-0353c6bc36c1 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:38.886292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113ab14-a37f-4319-837b-0353c6bc36c1 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:38.886575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c113ab14-a37f-4319-837b-0353c6bc36c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:38.888208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c113ab14-a37f-4319-837b-0353c6bc36c1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:38.894995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:38.977304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113ab14-a37f-4319-837b-0353c6bc36c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:38.978884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c113ab14-a37f-4319-837b-0353c6bc36c1 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:38.997421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 716/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:40.055360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ad079-122f-4484-9d5f-30e45e7463ff None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:40.055974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ad079-122f-4484-9d5f-30e45e7463ff None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:40.055974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063ad079-122f-4484-9d5f-30e45e7463ff None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:40.057256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:40.063780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401883c2-6a38-441f-a824-01e83d74cf5c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:40.064234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401883c2-6a38-441f-a824-01e83d74cf5c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:40.064360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401883c2-6a38-441f-a824-01e83d74cf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:40.141103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-401883c2-6a38-441f-a824-01e83d74cf5c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:40.142243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-401883c2-6a38-441f-a824-01e83d74cf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:40.209038 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:40.239124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:07:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:40.390168 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ad079-122f-4484-9d5f-30e45e7463ff None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:40.391825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-063ad079-122f-4484-9d5f-30e45e7463ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:40.409727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:40.538016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063ad079-122f-4484-9d5f-30e45e7463ff None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:40.625457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:40.651117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df26ff2-3fa8-4ce5-82b0-19c5a5baf53b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:40.652490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df26ff2-3fa8-4ce5-82b0-19c5a5baf53b service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:40.652781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df26ff2-3fa8-4ce5-82b0-19c5a5baf53b service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:40.653048 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df26ff2-3fa8-4ce5-82b0-19c5a5baf53b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:40.654620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1df26ff2-3fa8-4ce5-82b0-19c5a5baf53b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:40.743129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df26ff2-3fa8-4ce5-82b0-19c5a5baf53b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:40.744758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df26ff2-3fa8-4ce5-82b0-19c5a5baf53b service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:40.762779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 717/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:41.831753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f810f-342c-4633-a015-210b2b9da5e5 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:41.832320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f810f-342c-4633-a015-210b2b9da5e5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:41.832451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3f810f-342c-4633-a015-210b2b9da5e5 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:41.833719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:07:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:41.840728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94eafc-b4e5-4420-b658-e890636bbce8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:41.841244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94eafc-b4e5-4420-b658-e890636bbce8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:41.841359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94eafc-b4e5-4420-b658-e890636bbce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:41.917707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2f94eafc-b4e5-4420-b658-e890636bbce8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:41.919064 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2f94eafc-b4e5-4420-b658-e890636bbce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:41.959368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:42.015707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:42.165462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f810f-342c-4633-a015-210b2b9da5e5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:42.167113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bf3f810f-342c-4633-a015-210b2b9da5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:42.311683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3f810f-342c-4633-a015-210b2b9da5e5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:42.365119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:42.424420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d5e0bb-07b4-40e4-86ee-3af621a9f959 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:42.425031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5e0bb-07b4-40e4-86ee-3af621a9f959 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:42.425307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5e0bb-07b4-40e4-86ee-3af621a9f959 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:42.425578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d5e0bb-07b4-40e4-86ee-3af621a9f959 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:42.427169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-41d5e0bb-07b4-40e4-86ee-3af621a9f959 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:42.515647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5e0bb-07b4-40e4-86ee-3af621a9f959 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:42.517232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d5e0bb-07b4-40e4-86ee-3af621a9f959 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:42.535641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 719/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:43.596227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55b0f5-ca51-4397-b27d-565047adc3d4 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:43.596773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55b0f5-ca51-4397-b27d-565047adc3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:43.596773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc55b0f5-ca51-4397-b27d-565047adc3d4 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:43.598137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:43.605019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f9100-7142-4cb7-be25-80b38500661e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:43.605458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f9100-7142-4cb7-be25-80b38500661e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:43.605574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f9100-7142-4cb7-be25-80b38500661e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:43.681630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e75f9100-7142-4cb7-be25-80b38500661e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:43.683467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e75f9100-7142-4cb7-be25-80b38500661e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:43.685475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:43.779321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:43.930935 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55b0f5-ca51-4397-b27d-565047adc3d4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:43.933131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cc55b0f5-ca51-4397-b27d-565047adc3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:44.083226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc55b0f5-ca51-4397-b27d-565047adc3d4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:44.111676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:44.197403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b1988-2dbe-4558-85ea-b572b7466e7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:44.198027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b1988-2dbe-4558-85ea-b572b7466e7e service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:44.199008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b1988-2dbe-4558-85ea-b572b7466e7e service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:44.199279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b1988-2dbe-4558-85ea-b572b7466e7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:44.200894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-185b1988-2dbe-4558-85ea-b572b7466e7e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:44.288800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b1988-2dbe-4558-85ea-b572b7466e7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:44.290404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b1988-2dbe-4558-85ea-b572b7466e7e service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:44.308583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 720/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:46.391382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dfa11-6044-4d7c-a090-c5236601a5e8 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:46.391931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dfa11-6044-4d7c-a090-c5236601a5e8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:46.391931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dfa11-6044-4d7c-a090-c5236601a5e8 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:46.393233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:46.398551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efb50f-db12-40ad-8015-3b493e5a954b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:46.399027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efb50f-db12-40ad-8015-3b493e5a954b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:46.399137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00efb50f-db12-40ad-8015-3b493e5a954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:46.443204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:46.468728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-00efb50f-db12-40ad-8015-3b493e5a954b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:46.469938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-00efb50f-db12-40ad-8015-3b493e5a954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:46.565086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:46.851799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dfa11-6044-4d7c-a090-c5236601a5e8 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:46.853333 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-854dfa11-6044-4d7c-a090-c5236601a5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:46.955345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dfa11-6044-4d7c-a090-c5236601a5e8 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:46.987539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:47.029182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5da01-6c98-4b5e-b87a-09ec08c9aea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:47.029783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5da01-6c98-4b5e-b87a-09ec08c9aea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:47.030058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5da01-6c98-4b5e-b87a-09ec08c9aea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:47.030326 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5da01-6c98-4b5e-b87a-09ec08c9aea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:47.031934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-43e5da01-6c98-4b5e-b87a-09ec08c9aea1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:47.145664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5da01-6c98-4b5e-b87a-09ec08c9aea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:47.147017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5da01-6c98-4b5e-b87a-09ec08c9aea1 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:47.165071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 722/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:49.522209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107c66f-a101-4421-bc19-abad703e0522 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:49.522722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107c66f-a101-4421-bc19-abad703e0522 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:49.522722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f107c66f-a101-4421-bc19-abad703e0522 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:49.524117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:49.529422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47684-0121-4efc-bd22-58e8f7c5d357 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:49.529850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47684-0121-4efc-bd22-58e8f7c5d357 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:49.529945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c47684-0121-4efc-bd22-58e8f7c5d357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:49.595831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-84c47684-0121-4efc-bd22-58e8f7c5d357 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:49.597031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-84c47684-0121-4efc-bd22-58e8f7c5d357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:49.609845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:49.694835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:49.982383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107c66f-a101-4421-bc19-abad703e0522 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:49.983946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f107c66f-a101-4421-bc19-abad703e0522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:50.086734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f107c66f-a101-4421-bc19-abad703e0522 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:50.099471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:50.161357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5be052-574d-493c-a436-d224b5469968 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:50.161953 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5be052-574d-493c-a436-d224b5469968 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:50.162227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5be052-574d-493c-a436-d224b5469968 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:50.162491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5be052-574d-493c-a436-d224b5469968 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:50.164099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2e5be052-574d-493c-a436-d224b5469968 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:50.273109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5be052-574d-493c-a436-d224b5469968 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:50.274546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5be052-574d-493c-a436-d224b5469968 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:50.293747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 723/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:52.868636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1ff85-3934-4aa1-8e9b-1e6407f3c362 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:52.869175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1ff85-3934-4aa1-8e9b-1e6407f3c362 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:52.869175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1ff85-3934-4aa1-8e9b-1e6407f3c362 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:52.870783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:52.876191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdb93e-e412-4574-a217-d4314ab8c686 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:52.876609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdb93e-e412-4574-a217-d4314ab8c686 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:52.876743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebdb93e-e412-4574-a217-d4314ab8c686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:52.925347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:52.943210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2ebdb93e-e412-4574-a217-d4314ab8c686 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:52.944441 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2ebdb93e-e412-4574-a217-d4314ab8c686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:53.040406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:07:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:53.324838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1ff85-3934-4aa1-8e9b-1e6407f3c362 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:53.326353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-dfe1ff85-3934-4aa1-8e9b-1e6407f3c362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:53.428146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1ff85-3934-4aa1-8e9b-1e6407f3c362 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:53.469276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:53.505040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571bf41-e195-428c-87df-5d3e06fbf347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:53.505646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571bf41-e195-428c-87df-5d3e06fbf347 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:53.505923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571bf41-e195-428c-87df-5d3e06fbf347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:53.506191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571bf41-e195-428c-87df-5d3e06fbf347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:53.507811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2571bf41-e195-428c-87df-5d3e06fbf347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:53.617618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571bf41-e195-428c-87df-5d3e06fbf347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:53.618968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571bf41-e195-428c-87df-5d3e06fbf347 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:53.637651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 725/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:55.925516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313303e-05eb-44dd-a968-4ed0c737d8d4 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:55.926040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313303e-05eb-44dd-a968-4ed0c737d8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:55.926040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8313303e-05eb-44dd-a968-4ed0c737d8d4 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:55.927376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:07:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:55.932899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b3b70-9e9f-4d7d-b1de-dc026a0c0479 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:55.933341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b3b70-9e9f-4d7d-b1de-dc026a0c0479 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:55.933450 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5b3b70-9e9f-4d7d-b1de-dc026a0c0479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:55.999713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7f5b3b70-9e9f-4d7d-b1de-dc026a0c0479 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:56.000911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7f5b3b70-9e9f-4d7d-b1de-dc026a0c0479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:56.021443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:56.104911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:07:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:56.392636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313303e-05eb-44dd-a968-4ed0c737d8d4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:56.394164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8313303e-05eb-44dd-a968-4ed0c737d8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:56.497495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8313303e-05eb-44dd-a968-4ed0c737d8d4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:56.517960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:56.570235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba1d618-3b7e-46f5-9fa6-d1e06c943378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:56.570833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1d618-3b7e-46f5-9fa6-d1e06c943378 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:56.571106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1d618-3b7e-46f5-9fa6-d1e06c943378 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:56.571375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba1d618-3b7e-46f5-9fa6-d1e06c943378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:56.573506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5ba1d618-3b7e-46f5-9fa6-d1e06c943378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:56.681710 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1d618-3b7e-46f5-9fa6-d1e06c943378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:56.683190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba1d618-3b7e-46f5-9fa6-d1e06c943378 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:56.701009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 726/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:07:58.982143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c8771-1a7b-4523-beca-7295a11ae716 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:58.982760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c8771-1a7b-4523-beca-7295a11ae716 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:58.982760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30c8771-1a7b-4523-beca-7295a11ae716 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:58.984079 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:07:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:07:58.989292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a43889-c512-4842-b1c4-31ac2d13e986 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:58.989738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a43889-c512-4842-b1c4-31ac2d13e986 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:58.989849 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a43889-c512-4842-b1c4-31ac2d13e986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:59.052093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:59.055048 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a9a43889-c512-4842-b1c4-31ac2d13e986 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:07:59.056833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a9a43889-c512-4842-b1c4-31ac2d13e986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:59.160086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:07:59.443790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c8771-1a7b-4523-beca-7295a11ae716 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:07:59.445317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e30c8771-1a7b-4523-beca-7295a11ae716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:59.547956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30c8771-1a7b-4523-beca-7295a11ae716 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:07:59.596288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:07:59.622184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8403c8c-201e-4a32-9afb-75d67776bf37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:07:59.622788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8403c8c-201e-4a32-9afb-75d67776bf37 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:07:59.623066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8403c8c-201e-4a32-9afb-75d67776bf37 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:07:59.623336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8403c8c-201e-4a32-9afb-75d67776bf37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:07:59.624969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f8403c8c-201e-4a32-9afb-75d67776bf37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:07:59.733178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8403c8c-201e-4a32-9afb-75d67776bf37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:07:59.734531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8403c8c-201e-4a32-9afb-75d67776bf37 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:07:59.752648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 728/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:07:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:08:01.973954 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71401166-55ba-4910-b41c-b6add7f0932b None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:01.974574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71401166-55ba-4910-b41c-b6add7f0932b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:01.974574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71401166-55ba-4910-b41c-b6add7f0932b None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:01.976152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:08:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:01.982570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1280-c861-4c80-8905-75db1bd58b07 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:01.983014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1280-c861-4c80-8905-75db1bd58b07 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:01.983123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a1280-c861-4c80-8905-75db1bd58b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:02.048250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-766a1280-c861-4c80-8905-75db1bd58b07 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:02.049446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-766a1280-c861-4c80-8905-75db1bd58b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:02.078175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:02.151505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:08:02.435963 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71401166-55ba-4910-b41c-b6add7f0932b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:02.437512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-71401166-55ba-4910-b41c-b6add7f0932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:02.540800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71401166-55ba-4910-b41c-b6add7f0932b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:02.570477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:02.616180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8a5552-1ef0-49c5-a51b-bf98d925372a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:02.616783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a5552-1ef0-49c5-a51b-bf98d925372a service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:02.617059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a5552-1ef0-49c5-a51b-bf98d925372a service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:02.617358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a5552-1ef0-49c5-a51b-bf98d925372a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:02.618961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ea8a5552-1ef0-49c5-a51b-bf98d925372a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:02.727188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a5552-1ef0-49c5-a51b-bf98d925372a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:02.728684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8a5552-1ef0-49c5-a51b-bf98d925372a service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:02.747728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 729/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:08:04.949693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c896d11-47a0-42db-9169-4d2ec7ac4774 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:04.950287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c896d11-47a0-42db-9169-4d2ec7ac4774 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:04.950287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c896d11-47a0-42db-9169-4d2ec7ac4774 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:04.951738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:04.957162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc9080-a208-439a-9859-519bbe97bf9d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:04.957587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc9080-a208-439a-9859-519bbe97bf9d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:04.957691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cc9080-a208-439a-9859-519bbe97bf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:05.023031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b4cc9080-a208-439a-9859-519bbe97bf9d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:05.024265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b4cc9080-a208-439a-9859-519bbe97bf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:05.032661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:05.122254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:08:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:08:05.403048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c896d11-47a0-42db-9169-4d2ec7ac4774 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:05.404604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4c896d11-47a0-42db-9169-4d2ec7ac4774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:05.509329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c896d11-47a0-42db-9169-4d2ec7ac4774 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:05.566287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:05.584472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30b267f-c6c2-45a4-860e-8bc935e53406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:05.585070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b267f-c6c2-45a4-860e-8bc935e53406 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:05.585341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b267f-c6c2-45a4-860e-8bc935e53406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:05.585606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30b267f-c6c2-45a4-860e-8bc935e53406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:05.587167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d30b267f-c6c2-45a4-860e-8bc935e53406 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:05.693402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b267f-c6c2-45a4-860e-8bc935e53406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:05.694869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30b267f-c6c2-45a4-860e-8bc935e53406 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:05.713175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 731/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:08:07.933688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a67e28-a5dc-4b66-9e42-8e7dc1c41d37 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:07.934218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a67e28-a5dc-4b66-9e42-8e7dc1c41d37 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:07.934218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a67e28-a5dc-4b66-9e42-8e7dc1c41d37 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:07.935622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:07.940969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9569a9-5924-48e6-9110-306b1a3afebf None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:07.941390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9569a9-5924-48e6-9110-306b1a3afebf None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:07.941508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9569a9-5924-48e6-9110-306b1a3afebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:08.007137 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-1a9569a9-5924-48e6-9110-306b1a3afebf None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:08.008397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1a9569a9-5924-48e6-9110-306b1a3afebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:08.045873 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:08.103519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:08:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:08:08.390586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a67e28-a5dc-4b66-9e42-8e7dc1c41d37 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:08.392167 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b1a67e28-a5dc-4b66-9e42-8e7dc1c41d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:08.498755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a67e28-a5dc-4b66-9e42-8e7dc1c41d37 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:08.539694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:08.577983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46987201-1cde-428a-af43-c6c6d63d29db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:08.578564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46987201-1cde-428a-af43-c6c6d63d29db service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:08.578843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46987201-1cde-428a-af43-c6c6d63d29db service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:08.579111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46987201-1cde-428a-af43-c6c6d63d29db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:08.580739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-46987201-1cde-428a-af43-c6c6d63d29db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:08.690574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46987201-1cde-428a-af43-c6c6d63d29db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:08.692067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46987201-1cde-428a-af43-c6c6d63d29db service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:08.710209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 732/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:08:10.899822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37d3a8-3cb0-44f3-83f4-baf467c4c589 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:10.900410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37d3a8-3cb0-44f3-83f4-baf467c4c589 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:10.900410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f37d3a8-3cb0-44f3-83f4-baf467c4c589 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:10.902417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:08:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:10.907794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03373eae-f82a-44e8-bf4e-cedcc5899398 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:10.908226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03373eae-f82a-44e8-bf4e-cedcc5899398 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:10.908329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03373eae-f82a-44e8-bf4e-cedcc5899398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:10.973553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-03373eae-f82a-44e8-bf4e-cedcc5899398 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:10.974750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-03373eae-f82a-44e8-bf4e-cedcc5899398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:10.993395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:11.071667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:08:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:08:11.355953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37d3a8-3cb0-44f3-83f4-baf467c4c589 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:11.357491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4f37d3a8-3cb0-44f3-83f4-baf467c4c589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:11.463076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f37d3a8-3cb0-44f3-83f4-baf467c4c589 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:11.529656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:11.536308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c9afa-9485-4e82-af4b-6a36599c346d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:11.536915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c9afa-9485-4e82-af4b-6a36599c346d service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:11.537194 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c9afa-9485-4e82-af4b-6a36599c346d service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:11.537467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c9afa-9485-4e82-af4b-6a36599c346d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:11.539054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1e6c9afa-9485-4e82-af4b-6a36599c346d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:11.647623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c9afa-9485-4e82-af4b-6a36599c346d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:11.648969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c9afa-9485-4e82-af4b-6a36599c346d service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:11.666972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 734/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:08:13.844648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20b149-21f2-4e44-81e0-a255c4016f5a None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:13.845196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20b149-21f2-4e44-81e0-a255c4016f5a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:13.845196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20b149-21f2-4e44-81e0-a255c4016f5a None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:13.846494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:13.851980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ccbb9-a1ee-475c-a603-8921c8c22512 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:13.852428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ccbb9-a1ee-475c-a603-8921c8c22512 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:13.852540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ccbb9-a1ee-475c-a603-8921c8c22512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:13.918192 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-8a7ccbb9-a1ee-475c-a603-8921c8c22512 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:13.919380 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-8a7ccbb9-a1ee-475c-a603-8921c8c22512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:13.967625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:14.016034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:08:14.301296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20b149-21f2-4e44-81e0-a255c4016f5a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:14.302888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8f20b149-21f2-4e44-81e0-a255c4016f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:14.405338 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20b149-21f2-4e44-81e0-a255c4016f5a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:14.454238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:14.481766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3638a-9ae2-42f8-a998-8beeff7c1b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:14.482366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3638a-9ae2-42f8-a998-8beeff7c1b59 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:14.482644 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3638a-9ae2-42f8-a998-8beeff7c1b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:14.482916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3638a-9ae2-42f8-a998-8beeff7c1b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:14.484537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a7c3638a-9ae2-42f8-a998-8beeff7c1b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:14.592544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3638a-9ae2-42f8-a998-8beeff7c1b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:14.593906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3638a-9ae2-42f8-a998-8beeff7c1b59 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:14.612864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 735/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:08:24.014498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=22 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:24.610627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=66 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:34.640149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41697b23-8494-43fc-b810-a55a5fb7a13d None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:34.641035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41697b23-8494-43fc-b810-a55a5fb7a13d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:34.641035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41697b23-8494-43fc-b810-a55a5fb7a13d None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:34.641919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 736/1471] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:34.648162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9daf6c-31ba-454b-91a3-0040ee47c861 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:34.648652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9daf6c-31ba-454b-91a3-0040ee47c861 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:34.648768 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9daf6c-31ba-454b-91a3-0040ee47c861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:34.729486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-da9daf6c-31ba-454b-91a3-0040ee47c861 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:34.730704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-da9daf6c-31ba-454b-91a3-0040ee47c861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:34.852408 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 736/1472] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:08:35.216598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d364c3-0b0a-4125-a75c-8244fe469500 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:35.216881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d364c3-0b0a-4125-a75c-8244fe469500 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:35.217156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d364c3-0b0a-4125-a75c-8244fe469500 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:35.218495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 737/1473] 10.222.0.26 () {54 vars in 966 bytes} [Sat Sep 10 14:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:35.223651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ac4b-64ee-4150-87cb-eadc29778c7a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:35.223930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ac4b-64ee-4150-87cb-eadc29778c7a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:35.224203 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4018ac4b-64ee-4150-87cb-eadc29778c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:35.325283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4018ac4b-64ee-4150-87cb-eadc29778c7a None None] MFA Rules not processed for user `cf75b583add44b6b87bff5dc67080d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:35.326484 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4018ac4b-64ee-4150-87cb-eadc29778c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:35.346766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:35.409760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 737/1474] 10.222.0.26 () {58 vars in 1063 bytes} [Sat Sep 10 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:08:35.418599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d364c3-0b0a-4125-a75c-8244fe469500 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:35.420049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-95d364c3-0b0a-4125-a75c-8244fe469500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:35.523527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d364c3-0b0a-4125-a75c-8244fe469500 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:35.598267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dada3347-44db-40a6-9977-e83ea294e550 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf75b583add44b6b87bff5dc67080d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:35.598945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3347-44db-40a6-9977-e83ea294e550 service glance] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:35.599217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3347-44db-40a6-9977-e83ea294e550 service glance] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:35.599506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada3347-44db-40a6-9977-e83ea294e550 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:35.601086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-dada3347-44db-40a6-9977-e83ea294e550 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:35.660335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:35.715561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada3347-44db-40a6-9977-e83ea294e550 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:35.716917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dada3347-44db-40a6-9977-e83ea294e550 service glance] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:35.735866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 738/1475] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:08:37.987865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93f85-8c06-45e2-a217-791c1bab8ed8 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:37.988440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93f85-8c06-45e2-a217-791c1bab8ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:37.988440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b93f85-8c06-45e2-a217-791c1bab8ed8 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:37.989795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 738/1476] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:37.995597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f087bfe-bb1d-4b11-bcc2-91095e461cdd None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:37.996008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f087bfe-bb1d-4b11-bcc2-91095e461cdd None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:37.996147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f087bfe-bb1d-4b11-bcc2-91095e461cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:38.062748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-9f087bfe-bb1d-4b11-bcc2-91095e461cdd None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:38.063980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9f087bfe-bb1d-4b11-bcc2-91095e461cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:38.159242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 739/1477] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:08:38.435458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b93f85-8c06-45e2-a217-791c1bab8ed8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:38.437012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a6b93f85-8c06-45e2-a217-791c1bab8ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:38.574202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b93f85-8c06-45e2-a217-791c1bab8ed8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:38.624674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:38.670731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a0ddf7-269b-4cd3-9ca7-376b031a85e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:38.671354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0ddf7-269b-4cd3-9ca7-376b031a85e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:38.671669 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0ddf7-269b-4cd3-9ca7-376b031a85e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:38.671940 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a0ddf7-269b-4cd3-9ca7-376b031a85e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:38.672606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0ddf7-269b-4cd3-9ca7-376b031a85e6 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:38.674412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-a0a0ddf7-269b-4cd3-9ca7-376b031a85e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:08:38.674412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Sep 10 14:08:38.674412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: Sep 10 14:08:38.675612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a0ddf7-269b-4cd3-9ca7-376b031a85e6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:38.694723 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 739/1478] 10.222.0.26 () {56 vars in 1108 bytes} [Sat Sep 10 14:08:38 2022] GET /identity/v3/projects => generated 2553 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:08:40.873810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fec6d-0c87-4c12-b869-295a081ab3df None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:40.874384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fec6d-0c87-4c12-b869-295a081ab3df None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:40.874384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7fec6d-0c87-4c12-b869-295a081ab3df None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:40.875753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 740/1479] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:40.881107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a27ab-b8d1-4c40-8617-19a2c9e511e3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:40.881565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a27ab-b8d1-4c40-8617-19a2c9e511e3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:40.881671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6a27ab-b8d1-4c40-8617-19a2c9e511e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:40.948845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-dd6a27ab-b8d1-4c40-8617-19a2c9e511e3 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:40.950052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-dd6a27ab-b8d1-4c40-8617-19a2c9e511e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:41.046264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 740/1480] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:08:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:08:41.342604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fec6d-0c87-4c12-b869-295a081ab3df None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:41.344276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9e7fec6d-0c87-4c12-b869-295a081ab3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:41.355182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:41.490802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7fec6d-0c87-4c12-b869-295a081ab3df None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:41.568624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:41.601158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f736077-2565-4cbe-a212-6c68c9c6e3b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:41.601948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736077-2565-4cbe-a212-6c68c9c6e3b9 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:41.602221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736077-2565-4cbe-a212-6c68c9c6e3b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:41.602485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f736077-2565-4cbe-a212-6c68c9c6e3b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:41.604611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1f736077-2565-4cbe-a212-6c68c9c6e3b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:41.692757 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f736077-2565-4cbe-a212-6c68c9c6e3b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:41.694343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f736077-2565-4cbe-a212-6c68c9c6e3b9 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:41.711715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 741/1481] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:08:44.058898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0f0fe-1560-44ea-90fa-9d08072913d8 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:44.059466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0f0fe-1560-44ea-90fa-9d08072913d8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:44.059466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea0f0fe-1560-44ea-90fa-9d08072913d8 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:44.060810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 741/1482] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:44.066055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d9e02-5657-4717-9fa7-ed5585504b89 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:44.066524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d9e02-5657-4717-9fa7-ed5585504b89 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:44.066631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39d9e02-5657-4717-9fa7-ed5585504b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:44.132772 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e39d9e02-5657-4717-9fa7-ed5585504b89 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:44.133975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e39d9e02-5657-4717-9fa7-ed5585504b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:44.179687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:44.229544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 742/1483] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:08:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:08:44.526740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0f0fe-1560-44ea-90fa-9d08072913d8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:44.528313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5ea0f0fe-1560-44ea-90fa-9d08072913d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:44.551984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:44.631702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea0f0fe-1560-44ea-90fa-9d08072913d8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:44.705597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e83e6d-abed-4132-9ed2-647739f8799d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:44.706125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e83e6d-abed-4132-9ed2-647739f8799d service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:44.706400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e83e6d-abed-4132-9ed2-647739f8799d service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:44.706671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e83e6d-abed-4132-9ed2-647739f8799d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:44.708292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-78e83e6d-abed-4132-9ed2-647739f8799d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:44.783901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:44.818726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e83e6d-abed-4132-9ed2-647739f8799d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:44.820207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e83e6d-abed-4132-9ed2-647739f8799d service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:44.839350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 742/1484] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:08:47.055715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d460e-be28-434d-bda1-05acd3a97d8d None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:47.056641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d460e-be28-434d-bda1-05acd3a97d8d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:47.056641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d460e-be28-434d-bda1-05acd3a97d8d None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:47.057774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 743/1485] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:47.063328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb24fa7-f648-4b0e-928f-e297c75cbc9d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:47.063775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb24fa7-f648-4b0e-928f-e297c75cbc9d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:47.063909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb24fa7-f648-4b0e-928f-e297c75cbc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:47.129820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-cbb24fa7-f648-4b0e-928f-e297c75cbc9d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:47.131040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cbb24fa7-f648-4b0e-928f-e297c75cbc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:47.226194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 743/1486] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:08:47.522958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d460e-be28-434d-bda1-05acd3a97d8d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:47.524504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-928d460e-be28-434d-bda1-05acd3a97d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:47.631429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d460e-be28-434d-bda1-05acd3a97d8d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:47.703656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:47.705197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7e23af-d1c8-42c4-a65f-13a0757b5f8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:47.705881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e23af-d1c8-42c4-a65f-13a0757b5f8e service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:47.706353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e23af-d1c8-42c4-a65f-13a0757b5f8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:47.706862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7e23af-d1c8-42c4-a65f-13a0757b5f8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:47.708727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-dc7e23af-d1c8-42c4-a65f-13a0757b5f8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:47.817426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e23af-d1c8-42c4-a65f-13a0757b5f8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:47.818873 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7e23af-d1c8-42c4-a65f-13a0757b5f8e service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:47.837120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 744/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:08:50.150701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f70f9-ac41-4ee5-8097-52a2b95fe661 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:50.151228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f70f9-ac41-4ee5-8097-52a2b95fe661 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:50.151350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472f70f9-ac41-4ee5-8097-52a2b95fe661 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:50.152653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 744/1488] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:08:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:50.157899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a48cc-e193-4e46-b6fb-dfbd4f9677b6 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:50.158321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a48cc-e193-4e46-b6fb-dfbd4f9677b6 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:50.158451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a48cc-e193-4e46-b6fb-dfbd4f9677b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:50.228621 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-890a48cc-e193-4e46-b6fb-dfbd4f9677b6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:50.229908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-890a48cc-e193-4e46-b6fb-dfbd4f9677b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:50.281495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:50.325758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 745/1489] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:08:50.625317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f70f9-ac41-4ee5-8097-52a2b95fe661 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:50.626877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-472f70f9-ac41-4ee5-8097-52a2b95fe661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:50.662540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:50.731161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472f70f9-ac41-4ee5-8097-52a2b95fe661 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:50.803507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a42ce-b67f-4f15-a73f-4120ca9787c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:50.804110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a42ce-b67f-4f15-a73f-4120ca9787c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:50.804384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a42ce-b67f-4f15-a73f-4120ca9787c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:50.804655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a42ce-b67f-4f15-a73f-4120ca9787c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:50.806858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-026a42ce-b67f-4f15-a73f-4120ca9787c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:50.889905 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:50.919514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a42ce-b67f-4f15-a73f-4120ca9787c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:50.920967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a42ce-b67f-4f15-a73f-4120ca9787c1 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:50.939638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 745/1490] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:08:53.301897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2ebb-260c-4683-912f-3ae52e09dd11 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:53.302477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2ebb-260c-4683-912f-3ae52e09dd11 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:53.302477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687e2ebb-260c-4683-912f-3ae52e09dd11 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:53.303890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 746/1491] 10.222.0.26 () {54 vars in 927 bytes} [Sat Sep 10 14:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:08:53.310061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798adef-5541-44c6-845b-4e28922c48d4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:53.310485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798adef-5541-44c6-845b-4e28922c48d4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:53.310616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2798adef-5541-44c6-845b-4e28922c48d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:53.377068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2798adef-5541-44c6-845b-4e28922c48d4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:08:53.378284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2798adef-5541-44c6-845b-4e28922c48d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:53.475385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 746/1492] 10.222.0.26 () {58 vars in 1024 bytes} [Sat Sep 10 14:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:08:53.772112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e2ebb-260c-4683-912f-3ae52e09dd11 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:08:53.773670 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-687e2ebb-260c-4683-912f-3ae52e09dd11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:53.882060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687e2ebb-260c-4683-912f-3ae52e09dd11 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:08:53.958752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7598f275-6d57-4707-86ba-6be80a468431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:08:53.959362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598f275-6d57-4707-86ba-6be80a468431 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:08:53.959677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598f275-6d57-4707-86ba-6be80a468431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:08:53.959947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7598f275-6d57-4707-86ba-6be80a468431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:08:53.961542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7598f275-6d57-4707-86ba-6be80a468431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:08:53.970024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:08:54.070283 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598f275-6d57-4707-86ba-6be80a468431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:08:54.072309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7598f275-6d57-4707-86ba-6be80a468431 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:08:54.090924 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 747/1493] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:09:03.473874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=82 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:09:04.088451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=50 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:07.045526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb9228-aecb-4047-8de2-79a358a5f10c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:07.046127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb9228-aecb-4047-8de2-79a358a5f10c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:07.046127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb9228-aecb-4047-8de2-79a358a5f10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:07.123225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bfcb9228-aecb-4047-8de2-79a358a5f10c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:10:07.124469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bfcb9228-aecb-4047-8de2-79a358a5f10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:07.219132 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 747/1494] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:10:07.228042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb40f-ce01-4646-bfb2-431aceff79f8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:07.228547 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb40f-ce01-4646-bfb2-431aceff79f8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:07.228651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdb40f-ce01-4646-bfb2-431aceff79f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:07.302065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e2bdb40f-ce01-4646-bfb2-431aceff79f8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:10:07.303172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e2bdb40f-ce01-4646-bfb2-431aceff79f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:07.399680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 748/1495] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:10:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:10:07.404274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb9228-aecb-4047-8de2-79a358a5f10c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:10:07.405483 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bfcb9228-aecb-4047-8de2-79a358a5f10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:07.417307 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb9228-aecb-4047-8de2-79a358a5f10c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:10:07.479386 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:07.513378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:10:07.513981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:07.514258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:07.514526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:07.515157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:10:07.515465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:10:07.516498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:10:07.516498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Sep 10 14:10:07.516498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: Sep 10 14:10:07.517375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:10:07.528962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 748/1496] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:10:07.537232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdb40f-ce01-4646-bfb2-431aceff79f8 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:10:07.547223 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdb40f-ce01-4646-bfb2-431aceff79f8 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:10:07.608896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:07.644091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be82ab3-fc8c-4564-afce-b0b986961081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:10:07.644917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be82ab3-fc8c-4564-afce-b0b986961081 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:07.645188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be82ab3-fc8c-4564-afce-b0b986961081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:07.645453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be82ab3-fc8c-4564-afce-b0b986961081 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:07.646087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be82ab3-fc8c-4564-afce-b0b986961081 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:10:07.647817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: WARNING py.warnings [None req-6be82ab3-fc8c-4564-afce-b0b986961081 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:10:07.647817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: warnings.warn(msg) Sep 10 14:10:07.647817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: Sep 10 14:10:07.649682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be82ab3-fc8c-4564-afce-b0b986961081 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:10:07.706750 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 749/1497] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:10:07 2022] POST /identity/v3/projects => generated 379 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:10:07.710945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:10:07.713035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:07.746354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e3ea54-4540-45be-abb2-f2354c43749a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:10:07.839166 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a9c86-36bf-48e3-8ba7-af393eba8aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:10:07.839831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a9c86-36bf-48e3-8ba7-af393eba8aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:07.840103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a9c86-36bf-48e3-8ba7-af393eba8aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:07.840370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a9c86-36bf-48e3-8ba7-af393eba8aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:07.840980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a9c86-36bf-48e3-8ba7-af393eba8aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:10:07.841985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-ab4a9c86-36bf-48e3-8ba7-af393eba8aa0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:10:07.841985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Sep 10 14:10:07.841985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: Sep 10 14:10:07.843182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a9c86-36bf-48e3-8ba7-af393eba8aa0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:10:07.884988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 749/1498] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:10:07 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:10:07.889039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be82ab3-fc8c-4564-afce-b0b986961081 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:10:07.890616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6be82ab3-fc8c-4564-afce-b0b986961081 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:07.952966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be82ab3-fc8c-4564-afce-b0b986961081 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:10:08.035664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:08.047156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab865d7-a41e-4250-b47b-dcac3771a6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:10:08.047787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab865d7-a41e-4250-b47b-dcac3771a6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:08.048064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab865d7-a41e-4250-b47b-dcac3771a6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:08.048333 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab865d7-a41e-4250-b47b-dcac3771a6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:08.048969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab865d7-a41e-4250-b47b-dcac3771a6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:10:08.049977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: WARNING py.warnings [None req-5ab865d7-a41e-4250-b47b-dcac3771a6a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:10:08.049977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: warnings.warn(msg) Sep 10 14:10:08.049977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: Sep 10 14:10:08.050868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab865d7-a41e-4250-b47b-dcac3771a6a2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:10:08.063105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 750/1499] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:10:07 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:10:08.067163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a9c86-36bf-48e3-8ba7-af393eba8aa0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:10:08.068696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ab4a9c86-36bf-48e3-8ba7-af393eba8aa0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:08.080440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a9c86-36bf-48e3-8ba7-af393eba8aa0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:10:08.105980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:08.174355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934c632-7b6f-4683-b7d7-6c533262eec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:10:08.175072 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934c632-7b6f-4683-b7d7-6c533262eec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:08.175351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934c632-7b6f-4683-b7d7-6c533262eec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:08.175650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934c632-7b6f-4683-b7d7-6c533262eec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:08.176287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934c632-7b6f-4683-b7d7-6c533262eec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:10:08.177305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-a934c632-7b6f-4683-b7d7-6c533262eec9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:10:08.177305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Sep 10 14:10:08.177305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: Sep 10 14:10:08.178187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934c632-7b6f-4683-b7d7-6c533262eec9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:10:08.190996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 750/1500] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:10:08 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:10:08.195215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab865d7-a41e-4250-b47b-dcac3771a6a2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:10:08.197293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5ab865d7-a41e-4250-b47b-dcac3771a6a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:08.230044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab865d7-a41e-4250-b47b-dcac3771a6a2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:10:08.324735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc0ddb-3850-4a91-804b-449e04073ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:10:08.325411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc0ddb-3850-4a91-804b-449e04073ef3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:08.325663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc0ddb-3850-4a91-804b-449e04073ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:08.325937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc0ddb-3850-4a91-804b-449e04073ef3 admin admin] PATH_INFO: `/v3/projects/b7566437010b4589b3cbb5cb2a902781/users/bc632401a8ad4461b77ae78c4669b064/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:08.348276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc0ddb-3850-4a91-804b-449e04073ef3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7566437010b4589b3cbb5cb2a902781, user_id=bc632401a8ad4461b77ae78c4669b064, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:10:08.349300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: WARNING py.warnings [None req-66bc0ddb-3850-4a91-804b-449e04073ef3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:10:08.349300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: warnings.warn(msg) Sep 10 14:10:08.349300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: Sep 10 14:10:08.351254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc0ddb-3850-4a91-804b-449e04073ef3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:10:08.376152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 751/1501] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:10:08 2022] PUT /identity/v3/projects/b7566437010b4589b3cbb5cb2a902781/users/bc632401a8ad4461b77ae78c4669b064/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:10:08.389085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f343e2-32b0-4900-8c81-d599eea5663d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:08.389375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f343e2-32b0-4900-8c81-d599eea5663d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:08.389654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f343e2-32b0-4900-8c81-d599eea5663d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:08.448742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d9f343e2-32b0-4900-8c81-d599eea5663d None None] MFA Rules not processed for user `bc632401a8ad4461b77ae78c4669b064`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:10:08.449934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d9f343e2-32b0-4900-8c81-d599eea5663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:08.526809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 751/1502] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:10:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:10:08.541350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784649a-3ef9-43c2-b8ec-eb9c2974383d None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:08.541636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784649a-3ef9-43c2-b8ec-eb9c2974383d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:08.541906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d784649a-3ef9-43c2-b8ec-eb9c2974383d None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:08.543257 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 752/1503] 10.222.0.26 () {54 vars in 964 bytes} [Sat Sep 10 14:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:10:08.548103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66466d93-6eb2-47f3-88d9-7b4442b765ed None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:08.548389 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66466d93-6eb2-47f3-88d9-7b4442b765ed None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:08.548663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66466d93-6eb2-47f3-88d9-7b4442b765ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:08.582603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:08.746450 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-66466d93-6eb2-47f3-88d9-7b4442b765ed None None] MFA Rules not processed for user `fdc278f50e794e12a27d30fca9d131c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:10:08.747618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-66466d93-6eb2-47f3-88d9-7b4442b765ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:08.824043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 752/1504] 10.222.0.26 () {58 vars in 1061 bytes} [Sat Sep 10 14:10:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:10:08.832678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784649a-3ef9-43c2-b8ec-eb9c2974383d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:10:08.834224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d784649a-3ef9-43c2-b8ec-eb9c2974383d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:08.884289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:08.937581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d784649a-3ef9-43c2-b8ec-eb9c2974383d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:10:09.010489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ad808-f83b-4702-9509-aa1d7c1f0c61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:10:09.011010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ad808-f83b-4702-9509-aa1d7c1f0c61 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:09.011281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ad808-f83b-4702-9509-aa1d7c1f0c61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:09.011577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ad808-f83b-4702-9509-aa1d7c1f0c61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:09.013137 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b99ad808-f83b-4702-9509-aa1d7c1f0c61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:09.110620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ad808-f83b-4702-9509-aa1d7c1f0c61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:10:09.112106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ad808-f83b-4702-9509-aa1d7c1f0c61 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:10:09.115544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:09.130703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 753/1505] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:10:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:10:09.176707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fe20b-9170-4691-be01-60023d5a0f40 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:09.177141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fe20b-9170-4691-be01-60023d5a0f40 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:09.177238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fe20b-9170-4691-be01-60023d5a0f40 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:09.178485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 753/1506] 10.222.0.26 () {52 vars in 830 bytes} [Sat Sep 10 14:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:10:09.182227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ad808-f83b-4702-9509-aa1d7c1f0c61 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:10:09.183431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b99ad808-f83b-4702-9509-aa1d7c1f0c61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:09.282312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ad808-f83b-4702-9509-aa1d7c1f0c61 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:10:09.378241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:10:09.378865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:09.379135 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:09.379399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] PATH_INFO: `/v3/users/bc632401a8ad4461b77ae78c4669b064` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:09.393385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc632401a8ad4461b77ae78c4669b064)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:10:09.394415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: WARNING py.warnings [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:10:09.394415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: warnings.warn(msg) Sep 10 14:10:09.394415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: Sep 10 14:10:09.395675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:10:09.405960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:09.462569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc632401a8ad4461b77ae78c4669b064'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:10:09.463109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc632401a8ad4461b77ae78c4669b064'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:10:09.472183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc632401a8ad4461b77ae78c4669b064'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:10:09.487161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc632401a8ad4461b77ae78c4669b064'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:10:09.487645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc632401a8ad4461b77ae78c4669b064'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:10:09.515738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 754/1507] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:10:09 2022] DELETE /identity/v3/users/bc632401a8ad4461b77ae78c4669b064 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:10:09.520216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d56264-1763-4ff8-a41f-384f90e779e4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:09.520496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d56264-1763-4ff8-a41f-384f90e779e4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:09.520763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d56264-1763-4ff8-a41f-384f90e779e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:09.576866 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-59d56264-1763-4ff8-a41f-384f90e779e4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:10:09.577989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-59d56264-1763-4ff8-a41f-384f90e779e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:09.653595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:09.666255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 754/1508] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:10:09.675989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:10:09.677216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:09.781670 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde833cf-6c93-41f0-929c-fb9b68505e81 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:10:09.836342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:09.856344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a466bd-a5f0-48d7-88c4-f8a06845b8cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:10:09.856961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a466bd-a5f0-48d7-88c4-f8a06845b8cf service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:09.857240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a466bd-a5f0-48d7-88c4-f8a06845b8cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:09.857514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a466bd-a5f0-48d7-88c4-f8a06845b8cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:09.859114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-25a466bd-a5f0-48d7-88c4-f8a06845b8cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:09.966574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a466bd-a5f0-48d7-88c4-f8a06845b8cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:10:09.968037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a466bd-a5f0-48d7-88c4-f8a06845b8cf service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:10:09.986910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 755/1509] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:10:10.337131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d56264-1763-4ff8-a41f-384f90e779e4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:10:10.338388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-59d56264-1763-4ff8-a41f-384f90e779e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:10:10.349646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d56264-1763-4ff8-a41f-384f90e779e4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:10:10.450875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:10:10.451529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:10:10.451802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:10:10.452067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] PATH_INFO: `/v3/projects/b7566437010b4589b3cbb5cb2a902781` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:10:10.463106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7566437010b4589b3cbb5cb2a902781)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:10:10.464177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:10:10.464177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Sep 10 14:10:10.464177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: Sep 10 14:10:10.465365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:10:10.522148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7566437010b4589b3cbb5cb2a902781', 'request_id': 'req-04d299c2-7c95-4cc6-8c30-9f9fa1140397'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:10:10.530248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7566437010b4589b3cbb5cb2a902781', 'request_id': 'req-04d299c2-7c95-4cc6-8c30-9f9fa1140397'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:10:10.540564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7566437010b4589b3cbb5cb2a902781', 'request_id': 'req-04d299c2-7c95-4cc6-8c30-9f9fa1140397'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:10:10.569541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] The token cache is being invalidate because project b7566437010b4589b3cbb5cb2a902781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:10:10.569826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:10:10.570253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-04d299c2-7c95-4cc6-8c30-9f9fa1140397 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:10:10.571875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 755/1510] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:10:10 2022] DELETE /identity/v3/projects/b7566437010b4589b3cbb5cb2a902781 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:10:19.984671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=62 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:10:20.567399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:11:33.922193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1f929-f75c-4664-8ade-c15359b91200 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:33.922809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1f929-f75c-4664-8ade-c15359b91200 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:33.922809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f1f929-f75c-4664-8ade-c15359b91200 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:33.924126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 756/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:11:33.929253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d1964-b2ff-46ff-8598-576103a3d3c6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:33.929692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d1964-b2ff-46ff-8598-576103a3d3c6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:33.929793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45d1964-b2ff-46ff-8598-576103a3d3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:34.011973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f45d1964-b2ff-46ff-8598-576103a3d3c6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:11:34.013188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f45d1964-b2ff-46ff-8598-576103a3d3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:34.114426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 756/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:11:34.269379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1f929-f75c-4664-8ade-c15359b91200 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:11:34.271574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-45f1f929-f75c-4664-8ade-c15359b91200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:34.400343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f1f929-f75c-4664-8ade-c15359b91200 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:11:34.407347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229ac14-2385-49b3-9ada-939c22990b8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:11:34.408001 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229ac14-2385-49b3-9ada-939c22990b8b service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:34.408274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229ac14-2385-49b3-9ada-939c22990b8b service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:34.408541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229ac14-2385-49b3-9ada-939c22990b8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:34.417575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229ac14-2385-49b3-9ada-939c22990b8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:11:34.419203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229ac14-2385-49b3-9ada-939c22990b8b service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:11:34.437278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 757/1513] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:11:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:11:35.517418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ae5a0-92d4-43b3-b523-c941297e4978 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:35.517983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ae5a0-92d4-43b3-b523-c941297e4978 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:35.517983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ae5a0-92d4-43b3-b523-c941297e4978 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:35.519279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 757/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:11:35.525256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6995d2-0b92-488e-810c-b8ae7406bae4 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:35.525693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6995d2-0b92-488e-810c-b8ae7406bae4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:35.525803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6995d2-0b92-488e-810c-b8ae7406bae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:35.545541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-cc6995d2-0b92-488e-810c-b8ae7406bae4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:11:35.546766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-cc6995d2-0b92-488e-810c-b8ae7406bae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:35.554511 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 758/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:11:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:11:35.708405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ae5a0-92d4-43b3-b523-c941297e4978 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:11:35.715769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be2299-39d4-4aa0-8307-b1267cf449c2 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:35.716088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be2299-39d4-4aa0-8307-b1267cf449c2 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:35.716891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16be2299-39d4-4aa0-8307-b1267cf449c2 None None] PATH_INFO: `/v3` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:35.718206 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 759/1516] 10.222.0.18 () {54 vars in 925 bytes} [Sat Sep 10 14:11:35 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:11:35.723347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ae5a0-92d4-43b3-b523-c941297e4978 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:11:35.724505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a87019-3b89-4b63-9e3b-a1afbd208507 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:35.724848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a87019-3b89-4b63-9e3b-a1afbd208507 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:35.725186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a87019-3b89-4b63-9e3b-a1afbd208507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:35.729066 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83660d5-de53-4961-a44c-3a52a1fe3602 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:11:35.729659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83660d5-de53-4961-a44c-3a52a1fe3602 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:35.729930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83660d5-de53-4961-a44c-3a52a1fe3602 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:35.730197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83660d5-de53-4961-a44c-3a52a1fe3602 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:35.739497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83660d5-de53-4961-a44c-3a52a1fe3602 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:11:35.741080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83660d5-de53-4961-a44c-3a52a1fe3602 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:11:35.751587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 758/1517] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:11:35.788781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a6a87019-3b89-4b63-9e3b-a1afbd208507 None None] MFA Rules not processed for user `2a6a70a10cfa49348cc8b2b42870909d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:11:35.790130 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a6a87019-3b89-4b63-9e3b-a1afbd208507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:35.856876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:11:35.878280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 760/1518] 10.222.0.18 () {58 vars in 1015 bytes} [Sat Sep 10 14:11:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:11:44.891818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647e4ad-2ebc-4b8c-90cc-55ac778d7202 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:44.892301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647e4ad-2ebc-4b8c-90cc-55ac778d7202 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:44.892417 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d647e4ad-2ebc-4b8c-90cc-55ac778d7202 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:44.893672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 759/1519] 10.222.0.26 () {54 vars in 961 bytes} [Sat Sep 10 14:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:11:44.898505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282438a6-f1e0-4fad-b71e-745f6633d632 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:44.898920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282438a6-f1e0-4fad-b71e-745f6633d632 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:44.899099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282438a6-f1e0-4fad-b71e-745f6633d632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:44.918975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-282438a6-f1e0-4fad-b71e-745f6633d632 None None] MFA Rules not processed for user `2a6a70a10cfa49348cc8b2b42870909d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:11:44.921129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-282438a6-f1e0-4fad-b71e-745f6633d632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:44.929629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 761/1520] 10.222.0.26 () {58 vars in 1058 bytes} [Sat Sep 10 14:11:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:11:44.937547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647e4ad-2ebc-4b8c-90cc-55ac778d7202 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:11:44.950879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d647e4ad-2ebc-4b8c-90cc-55ac778d7202 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:11:44.955887 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b398455c-5684-4a77-95be-4ac4252b2191 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:11:44.956469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398455c-5684-4a77-95be-4ac4252b2191 service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:44.956743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398455c-5684-4a77-95be-4ac4252b2191 service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:44.957014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398455c-5684-4a77-95be-4ac4252b2191 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:44.965102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398455c-5684-4a77-95be-4ac4252b2191 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:11:44.967422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b398455c-5684-4a77-95be-4ac4252b2191 service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:11:44.976886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 760/1521] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:11:47.250874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ed1a-ad5b-4e74-88a3-82b391212a67 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:47.251472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ed1a-ad5b-4e74-88a3-82b391212a67 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:47.251472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e284ed1a-ad5b-4e74-88a3-82b391212a67 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:47.252851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:11:47.259058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450b61d-ba56-4397-8340-9515bd62cede None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:47.259476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450b61d-ba56-4397-8340-9515bd62cede None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:47.259632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9450b61d-ba56-4397-8340-9515bd62cede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:47.280013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9450b61d-ba56-4397-8340-9515bd62cede None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:11:47.281198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9450b61d-ba56-4397-8340-9515bd62cede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:47.288973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:11:47.442254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ed1a-ad5b-4e74-88a3-82b391212a67 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:11:47.457328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e284ed1a-ad5b-4e74-88a3-82b391212a67 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:11:47.462643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715b1fa-b02b-48e1-8abb-d42bbf4997e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:11:47.463311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b1fa-b02b-48e1-8abb-d42bbf4997e2 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:47.463621 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b1fa-b02b-48e1-8abb-d42bbf4997e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:47.463889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b1fa-b02b-48e1-8abb-d42bbf4997e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:47.472320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715b1fa-b02b-48e1-8abb-d42bbf4997e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:11:47.474029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715b1fa-b02b-48e1-8abb-d42bbf4997e2 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:11:47.484297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 763/1524] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:11:48.553475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e736c-762f-4c65-aaf9-bebb5c3964e5 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:48.554016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e736c-762f-4c65-aaf9-bebb5c3964e5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:48.554016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e736c-762f-4c65-aaf9-bebb5c3964e5 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:48.555342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sat Sep 10 14:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:11:48.562247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ec10-829b-4d28-a25b-1c5daed8d689 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:48.562672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ec10-829b-4d28-a25b-1c5daed8d689 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:48.562776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ec10-829b-4d28-a25b-1c5daed8d689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:48.583897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-6156ec10-829b-4d28-a25b-1c5daed8d689 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:11:48.585128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6156ec10-829b-4d28-a25b-1c5daed8d689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:48.593494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sat Sep 10 14:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:11:48.748677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e736c-762f-4c65-aaf9-bebb5c3964e5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:11:48.763623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e736c-762f-4c65-aaf9-bebb5c3964e5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:11:48.769247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4329b8fa-8782-44d5-9244-e66b5b618380 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:11:48.769838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329b8fa-8782-44d5-9244-e66b5b618380 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:48.770116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329b8fa-8782-44d5-9244-e66b5b618380 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:48.770394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4329b8fa-8782-44d5-9244-e66b5b618380 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:48.778284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329b8fa-8782-44d5-9244-e66b5b618380 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:11:48.780148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4329b8fa-8782-44d5-9244-e66b5b618380 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:11:48.789439 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 763/1527] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:11:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:11:52.840660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daae92-30e3-4d29-b25a-520c87f6b2ad None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:52.841243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daae92-30e3-4d29-b25a-520c87f6b2ad None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:52.841243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5daae92-30e3-4d29-b25a-520c87f6b2ad None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:52.842608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:11:52.848031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d90ffe-bf06-435c-9f57-3f2a020d887b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:52.848466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d90ffe-bf06-435c-9f57-3f2a020d887b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:52.848571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d90ffe-bf06-435c-9f57-3f2a020d887b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:52.868277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f3d90ffe-bf06-435c-9f57-3f2a020d887b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:11:52.869478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f3d90ffe-bf06-435c-9f57-3f2a020d887b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:52.877875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:11:53.163967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daae92-30e3-4d29-b25a-520c87f6b2ad None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:11:53.165640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c5daae92-30e3-4d29-b25a-520c87f6b2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:53.243929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5daae92-30e3-4d29-b25a-520c87f6b2ad None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:11:53.247632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47bad85-fe01-4657-b8f9-9bed275ce5bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:11:53.248321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bad85-fe01-4657-b8f9-9bed275ce5bf service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:53.248594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bad85-fe01-4657-b8f9-9bed275ce5bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:53.248863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47bad85-fe01-4657-b8f9-9bed275ce5bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:53.257521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bad85-fe01-4657-b8f9-9bed275ce5bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:11:53.258963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47bad85-fe01-4657-b8f9-9bed275ce5bf service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:11:53.268387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 766/1530] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:11:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:11:55.436839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8d421-0b47-44bc-9e99-aaa2fe883e2e None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:55.437447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8d421-0b47-44bc-9e99-aaa2fe883e2e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:55.437447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8d421-0b47-44bc-9e99-aaa2fe883e2e None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:55.438721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:11:55.444679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb2567-baef-4c57-875f-6a767859df33 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:55.445106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb2567-baef-4c57-875f-6a767859df33 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:55.445211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb2567-baef-4c57-875f-6a767859df33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:55.465077 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-10fb2567-baef-4c57-875f-6a767859df33 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:11:55.466543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-10fb2567-baef-4c57-875f-6a767859df33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:55.474297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:11:55.770699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8d421-0b47-44bc-9e99-aaa2fe883e2e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:11:55.772429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-32e8d421-0b47-44bc-9e99-aaa2fe883e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:55.785123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8d421-0b47-44bc-9e99-aaa2fe883e2e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:11:55.788548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9805847-5bb4-41b4-8354-00a40ec247ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:11:55.789124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9805847-5bb4-41b4-8354-00a40ec247ab service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:55.789395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9805847-5bb4-41b4-8354-00a40ec247ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:55.789661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9805847-5bb4-41b4-8354-00a40ec247ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:55.799258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9805847-5bb4-41b4-8354-00a40ec247ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:11:55.800721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9805847-5bb4-41b4-8354-00a40ec247ab service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:11:55.811652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 766/1533] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:11:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:11:59.359837 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c32f74-2673-4e5d-b798-dc2d63df518f None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:59.360383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c32f74-2673-4e5d-b798-dc2d63df518f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:59.360383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c32f74-2673-4e5d-b798-dc2d63df518f None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:59.361717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:11:59.366897 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0484f34-a6ac-4902-8c8d-d6c0a7ca488a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:59.367325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0484f34-a6ac-4902-8c8d-d6c0a7ca488a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:59.367425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0484f34-a6ac-4902-8c8d-d6c0a7ca488a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:59.375090 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:11:59.387428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d0484f34-a6ac-4902-8c8d-d6c0a7ca488a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:11:59.389159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d0484f34-a6ac-4902-8c8d-d6c0a7ca488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:59.396973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:11:59.701454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c32f74-2673-4e5d-b798-dc2d63df518f None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:11:59.703133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-20c32f74-2673-4e5d-b798-dc2d63df518f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:11:59.780915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c32f74-2673-4e5d-b798-dc2d63df518f None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:11:59.784452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988c3a7-2be7-40f0-afac-58c09b31ee8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:11:59.785052 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988c3a7-2be7-40f0-afac-58c09b31ee8c service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:11:59.785339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988c3a7-2be7-40f0-afac-58c09b31ee8c service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:11:59.785609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988c3a7-2be7-40f0-afac-58c09b31ee8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:11:59.795026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988c3a7-2be7-40f0-afac-58c09b31ee8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:11:59.796487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988c3a7-2be7-40f0-afac-58c09b31ee8c service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:11:59.805637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 769/1536] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:12:01.960811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e30c0-98b6-4458-968f-39270acf9a4d None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:12:01.961384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e30c0-98b6-4458-968f-39270acf9a4d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:12:01.961384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5e30c0-98b6-4458-968f-39270acf9a4d None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:12:01.962750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sat Sep 10 14:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:12:01.968825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ba35-2918-49b8-b19c-dc6440b922b4 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:12:01.969246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ba35-2918-49b8-b19c-dc6440b922b4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:12:01.969368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5ba35-2918-49b8-b19c-dc6440b922b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:12:01.989542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-52d5ba35-2918-49b8-b19c-dc6440b922b4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:12:01.990719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-52d5ba35-2918-49b8-b19c-dc6440b922b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:12:01.998680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sat Sep 10 14:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:12:02.302943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e30c0-98b6-4458-968f-39270acf9a4d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:12:02.304660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cd5e30c0-98b6-4458-968f-39270acf9a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:12:02.317457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5e30c0-98b6-4458-968f-39270acf9a4d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:12:02.320910 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a2b298-3c56-48a5-a071-77c3a93ba20e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:12:02.321491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2b298-3c56-48a5-a071-77c3a93ba20e service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:12:02.321765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2b298-3c56-48a5-a071-77c3a93ba20e service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:12:02.322040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a2b298-3c56-48a5-a071-77c3a93ba20e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:12:02.331569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2b298-3c56-48a5-a071-77c3a93ba20e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:12:02.333004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a2b298-3c56-48a5-a071-77c3a93ba20e service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:12:02.342971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 769/1539] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:12:11.984839 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:12:12.338137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=8 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:26.303566 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdccdc-f231-4f23-b7ca-6faeb1f223ec None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.304298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdccdc-f231-4f23-b7ca-6faeb1f223ec None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.304298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdccdc-f231-4f23-b7ca-6faeb1f223ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.331825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7ffdccdc-f231-4f23-b7ca-6faeb1f223ec None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.333028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7ffdccdc-f231-4f23-b7ca-6faeb1f223ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.346782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 771/1540] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.355793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b170573-0ab7-411a-b784-49ff0057283a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.356917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b170573-0ab7-411a-b784-49ff0057283a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.358283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b170573-0ab7-411a-b784-49ff0057283a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.387918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0b170573-0ab7-411a-b784-49ff0057283a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.390729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0b170573-0ab7-411a-b784-49ff0057283a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.403859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 770/1541] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.408877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdccdc-f231-4f23-b7ca-6faeb1f223ec None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:26.422433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdccdc-f231-4f23-b7ca-6faeb1f223ec None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:26.438087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc84ddf-9689-443b-a718-5a617548ef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:26.438724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc84ddf-9689-443b-a718-5a617548ef9c admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.439004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc84ddf-9689-443b-a718-5a617548ef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.439275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc84ddf-9689-443b-a718-5a617548ef9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.440356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc84ddf-9689-443b-a718-5a617548ef9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:26.440356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc84ddf-9689-443b-a718-5a617548ef9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:26.441375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: WARNING py.warnings [None req-1bc84ddf-9689-443b-a718-5a617548ef9c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:13:26.441375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: warnings.warn(msg) Sep 10 14:13:26.441375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: Sep 10 14:13:26.442317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc84ddf-9689-443b-a718-5a617548ef9c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:26.463874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 772/1542] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:26.472912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b170573-0ab7-411a-b784-49ff0057283a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:26.485301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b170573-0ab7-411a-b784-49ff0057283a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:26.496460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b613a6e-3d63-4ae7-9b82-a1c8eaad1165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:26.497449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b613a6e-3d63-4ae7-9b82-a1c8eaad1165 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.498723 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b613a6e-3d63-4ae7-9b82-a1c8eaad1165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.499771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b613a6e-3d63-4ae7-9b82-a1c8eaad1165 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.500636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b613a6e-3d63-4ae7-9b82-a1c8eaad1165 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:26.502733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-0b613a6e-3d63-4ae7-9b82-a1c8eaad1165 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:13:26.502733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Sep 10 14:13:26.502733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: Sep 10 14:13:26.504466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b613a6e-3d63-4ae7-9b82-a1c8eaad1165 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:26.566185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 771/1543] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:26.567953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c73e5e-5435-4b82-b4f7-7223ce87794e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.568238 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c73e5e-5435-4b82-b4f7-7223ce87794e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.568510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c73e5e-5435-4b82-b4f7-7223ce87794e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.575297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b613a6e-3d63-4ae7-9b82-a1c8eaad1165 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:26.606497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-00c73e5e-5435-4b82-b4f7-7223ce87794e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.608151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-00c73e5e-5435-4b82-b4f7-7223ce87794e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.661324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b613a6e-3d63-4ae7-9b82-a1c8eaad1165 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:26.667314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1847293a-b507-4e71-bbe0-8d1f8c5b50e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:26.668365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847293a-b507-4e71-bbe0-8d1f8c5b50e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.668861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847293a-b507-4e71-bbe0-8d1f8c5b50e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.669381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1847293a-b507-4e71-bbe0-8d1f8c5b50e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.671887 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847293a-b507-4e71-bbe0-8d1f8c5b50e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:26.675622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1847293a-b507-4e71-bbe0-8d1f8c5b50e1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:26.710876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 772/1544] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:13:26.713058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2321d7f-ac16-4e82-a754-6b1e103a5c96 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.713571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2321d7f-ac16-4e82-a754-6b1e103a5c96 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.715142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2321d7f-ac16-4e82-a754-6b1e103a5c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.722414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 773/1545] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.724088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd6a4-db58-42a5-bbbe-1a532cc4c4fd None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.724425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd6a4-db58-42a5-bbbe-1a532cc4c4fd None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.724712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ebd6a4-db58-42a5-bbbe-1a532cc4c4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.743065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d2321d7f-ac16-4e82-a754-6b1e103a5c96 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.744673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d2321d7f-ac16-4e82-a754-6b1e103a5c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.754932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 773/1546] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.756919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2321d7f-ac16-4e82-a754-6b1e103a5c96 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:26.760616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-b1ebd6a4-db58-42a5-bbbe-1a532cc4c4fd None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.761832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b1ebd6a4-db58-42a5-bbbe-1a532cc4c4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.770214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2321d7f-ac16-4e82-a754-6b1e103a5c96 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:26.775573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19af2f56-0426-472d-aa92-bb2ff2b439b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:26.777236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af2f56-0426-472d-aa92-bb2ff2b439b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.777791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af2f56-0426-472d-aa92-bb2ff2b439b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.778287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19af2f56-0426-472d-aa92-bb2ff2b439b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.779886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af2f56-0426-472d-aa92-bb2ff2b439b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:26.780234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 774/1547] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.781494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c955385-4991-4b2a-98f3-95e737e95379 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.781780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c955385-4991-4b2a-98f3-95e737e95379 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.782060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c955385-4991-4b2a-98f3-95e737e95379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.782968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19af2f56-0426-472d-aa92-bb2ff2b439b2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:26.797581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 774/1548] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:26.799242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09199a-a142-469c-bc3c-37ef4d87b71b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.799890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09199a-a142-469c-bc3c-37ef4d87b71b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.800394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b09199a-a142-469c-bc3c-37ef4d87b71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.805351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-6c955385-4991-4b2a-98f3-95e737e95379 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.806565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6c955385-4991-4b2a-98f3-95e737e95379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.815291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 775/1549] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.816716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65f4a-33eb-41a0-a8b1-d18596ae4000 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.817119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65f4a-33eb-41a0-a8b1-d18596ae4000 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.817417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e65f4a-33eb-41a0-a8b1-d18596ae4000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.827504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4b09199a-a142-469c-bc3c-37ef4d87b71b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.829535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4b09199a-a142-469c-bc3c-37ef4d87b71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.843205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 775/1550] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.843546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-76e65f4a-33eb-41a0-a8b1-d18596ae4000 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.844758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-76e65f4a-33eb-41a0-a8b1-d18596ae4000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.845410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e4085-d141-4d62-a324-77547a800c19 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.845686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e4085-d141-4d62-a324-77547a800c19 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.845950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e4085-d141-4d62-a324-77547a800c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.854609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 776/1551] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.856102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c5aae-b0ea-49b2-a92d-7748bd315c94 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.856416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c5aae-b0ea-49b2-a92d-7748bd315c94 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.856689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c5aae-b0ea-49b2-a92d-7748bd315c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.866926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c94e4085-d141-4d62-a324-77547a800c19 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.868137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c94e4085-d141-4d62-a324-77547a800c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.877345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 776/1552] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.877639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-740c5aae-b0ea-49b2-a92d-7748bd315c94 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.878186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e4085-d141-4d62-a324-77547a800c19 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:26.878811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-740c5aae-b0ea-49b2-a92d-7748bd315c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.886957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 777/1553] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.887848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e4085-d141-4d62-a324-77547a800c19 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:26.887970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c5aae-b0ea-49b2-a92d-7748bd315c94 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:26.893692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c02818-161f-4f39-afc1-5d8405ace4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:26.894291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02818-161f-4f39-afc1-5d8405ace4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.894585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02818-161f-4f39-afc1-5d8405ace4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.894820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c02818-161f-4f39-afc1-5d8405ace4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.895476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c02818-161f-4f39-afc1-5d8405ace4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:26.897084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c02818-161f-4f39-afc1-5d8405ace4e0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:26.897775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c5aae-b0ea-49b2-a92d-7748bd315c94 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:26.902316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e238b-3c0f-4573-859c-4c08fd0c7ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:26.902903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e238b-3c0f-4573-859c-4c08fd0c7ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.903180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e238b-3c0f-4573-859c-4c08fd0c7ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.903462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e238b-3c0f-4573-859c-4c08fd0c7ca0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.904173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e238b-3c0f-4573-859c-4c08fd0c7ca0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:26.904471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e238b-3c0f-4573-859c-4c08fd0c7ca0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:26.906064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e238b-3c0f-4573-859c-4c08fd0c7ca0 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:26.908337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 777/1554] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:26.909612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced054d-4109-4d4e-89cb-fda319410cf8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.909895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced054d-4109-4d4e-89cb-fda319410cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.910663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced054d-4109-4d4e-89cb-fda319410cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.921259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 778/1555] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:26.922348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b26d61-6a5c-434e-bfc4-60a2432986a7 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.922632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b26d61-6a5c-434e-bfc4-60a2432986a7 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.922903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b26d61-6a5c-434e-bfc4-60a2432986a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.930651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-aced054d-4109-4d4e-89cb-fda319410cf8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.931869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-aced054d-4109-4d4e-89cb-fda319410cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.940294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 778/1556] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.941582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843342d-d8ac-485f-bf34-f8cacfa00669 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.941865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843342d-d8ac-485f-bf34-f8cacfa00669 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.942128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-25b26d61-6a5c-434e-bfc4-60a2432986a7 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.942246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5843342d-d8ac-485f-bf34-f8cacfa00669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.943303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-25b26d61-6a5c-434e-bfc4-60a2432986a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.951743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 779/1557] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.952524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b26d61-6a5c-434e-bfc4-60a2432986a7 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:26.961896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b26d61-6a5c-434e-bfc4-60a2432986a7 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:26.966493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb9e90-7ad6-4593-9256-b2e84905c4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:26.967105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb9e90-7ad6-4593-9256-b2e84905c4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.967383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb9e90-7ad6-4593-9256-b2e84905c4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.967687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccb9e90-7ad6-4593-9256-b2e84905c4b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.968270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5843342d-d8ac-485f-bf34-f8cacfa00669 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:26.968418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb9e90-7ad6-4593-9256-b2e84905c4b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:26.968615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb9e90-7ad6-4593-9256-b2e84905c4b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:26.969578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5843342d-d8ac-485f-bf34-f8cacfa00669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:26.970222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccb9e90-7ad6-4593-9256-b2e84905c4b7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:26.977632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 779/1558] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:26.980984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd811d98-5a41-49aa-92bb-17f9367a6cdc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:26.981407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd811d98-5a41-49aa-92bb-17f9367a6cdc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:26.981764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd811d98-5a41-49aa-92bb-17f9367a6cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:26.982887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 780/1559] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:26.983743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb9e90-7ad6-4593-9256-b2e84905c4b7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:26.993877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccb9e90-7ad6-4593-9256-b2e84905c4b7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:26.999542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa68a2-9039-47e2-87cf-6babf5b5453d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.000129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa68a2-9039-47e2-87cf-6babf5b5453d admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.000404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa68a2-9039-47e2-87cf-6babf5b5453d admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.000675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaa68a2-9039-47e2-87cf-6babf5b5453d admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.001302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa68a2-9039-47e2-87cf-6babf5b5453d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:27.001598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa68a2-9039-47e2-87cf-6babf5b5453d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.002768 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fd811d98-5a41-49aa-92bb-17f9367a6cdc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:27.003182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaa68a2-9039-47e2-87cf-6babf5b5453d admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.003977 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fd811d98-5a41-49aa-92bb-17f9367a6cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:27.011704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 780/1560] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:27.012448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd811d98-5a41-49aa-92bb-17f9367a6cdc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.025623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 781/1561] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:27.025623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa68a2-9039-47e2-87cf-6babf5b5453d admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.025869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd811d98-5a41-49aa-92bb-17f9367a6cdc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.026151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a09a45-43f2-4b2f-968d-67b3ed5b4801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.026794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a09a45-43f2-4b2f-968d-67b3ed5b4801 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.027066 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a09a45-43f2-4b2f-968d-67b3ed5b4801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.027336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a09a45-43f2-4b2f-968d-67b3ed5b4801 admin admin] PATH_INFO: `/v3/projects/68f11e95e757488fa7c4c156194d24ff/users/db2f1171c5394a1c8879cd09c1e685b1/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.031293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaa68a2-9039-47e2-87cf-6babf5b5453d admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.035819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef835c0b-462e-4a6f-893b-b4c4c56b1730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.036394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef835c0b-462e-4a6f-893b-b4c4c56b1730 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.036669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef835c0b-462e-4a6f-893b-b4c4c56b1730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.036935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef835c0b-462e-4a6f-893b-b4c4c56b1730 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.037550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef835c0b-462e-4a6f-893b-b4c4c56b1730 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.039433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef835c0b-462e-4a6f-893b-b4c4c56b1730 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.042118 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a09a45-43f2-4b2f-968d-67b3ed5b4801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f11e95e757488fa7c4c156194d24ff, user_id=db2f1171c5394a1c8879cd09c1e685b1, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.043147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-30a09a45-43f2-4b2f-968d-67b3ed5b4801 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:13:27.043147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Sep 10 14:13:27.043147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: Sep 10 14:13:27.045714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a09a45-43f2-4b2f-968d-67b3ed5b4801 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.059571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 781/1562] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:27 2022] PUT /identity/v3/projects/68f11e95e757488fa7c4c156194d24ff/users/db2f1171c5394a1c8879cd09c1e685b1/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:27.061470 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65190a-69ad-4b21-8fd8-421e399a3ad4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.061861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65190a-69ad-4b21-8fd8-421e399a3ad4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.062207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a65190a-69ad-4b21-8fd8-421e399a3ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.074917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 782/1563] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/projects => generated 379 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.074917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84d4d3-bf0d-4711-81c1-19d4488aafd6 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.074917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84d4d3-bf0d-4711-81c1-19d4488aafd6 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.074917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84d4d3-bf0d-4711-81c1-19d4488aafd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.082267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0a65190a-69ad-4b21-8fd8-421e399a3ad4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:27.083501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0a65190a-69ad-4b21-8fd8-421e399a3ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:27.093373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-8e84d4d3-bf0d-4711-81c1-19d4488aafd6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:27.094532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-8e84d4d3-bf0d-4711-81c1-19d4488aafd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:27.103310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:27.139393 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:27.146334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 782/1564] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:27.147568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65190a-69ad-4b21-8fd8-421e399a3ad4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.157050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a65190a-69ad-4b21-8fd8-421e399a3ad4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.158960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 783/1565] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:27.159785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84d4d3-bf0d-4711-81c1-19d4488aafd6 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.162192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff73dc-b589-4b12-a00e-1e2b1ca47f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.162882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff73dc-b589-4b12-a00e-1e2b1ca47f7b admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.163162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff73dc-b589-4b12-a00e-1e2b1ca47f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.163431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff73dc-b589-4b12-a00e-1e2b1ca47f7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.164112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff73dc-b589-4b12-a00e-1e2b1ca47f7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:27.164408 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff73dc-b589-4b12-a00e-1e2b1ca47f7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.166005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff73dc-b589-4b12-a00e-1e2b1ca47f7b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.169162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84d4d3-bf0d-4711-81c1-19d4488aafd6 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.174400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21bba5-c1ab-496f-ad40-af055331d34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.174994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21bba5-c1ab-496f-ad40-af055331d34c admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.175270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21bba5-c1ab-496f-ad40-af055331d34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.175566 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21bba5-c1ab-496f-ad40-af055331d34c admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.176312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21bba5-c1ab-496f-ad40-af055331d34c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.178266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae21bba5-c1ab-496f-ad40-af055331d34c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.178748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 783/1566] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:27.179549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff73dc-b589-4b12-a00e-1e2b1ca47f7b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.192324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff73dc-b589-4b12-a00e-1e2b1ca47f7b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.196889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6f606b-4ed3-477f-8f3a-cc82a76ded7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.197543 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f606b-4ed3-477f-8f3a-cc82a76ded7e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.197866 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f606b-4ed3-477f-8f3a-cc82a76ded7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.198086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6f606b-4ed3-477f-8f3a-cc82a76ded7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.198798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f606b-4ed3-477f-8f3a-cc82a76ded7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:27.199091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f606b-4ed3-477f-8f3a-cc82a76ded7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.200709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6f606b-4ed3-477f-8f3a-cc82a76ded7e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.207569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 784/1567] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.208471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21bba5-c1ab-496f-ad40-af055331d34c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.215545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 784/1568] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:27.223578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6188e3-0c37-47a1-86f6-3c49ab1960d6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.223578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6188e3-0c37-47a1-86f6-3c49ab1960d6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.223578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6188e3-0c37-47a1-86f6-3c49ab1960d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.238022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-be6188e3-0c37-47a1-86f6-3c49ab1960d6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:27.239217 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-be6188e3-0c37-47a1-86f6-3c49ab1960d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:27.271496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21bba5-c1ab-496f-ad40-af055331d34c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.276299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b46b4-893f-46d9-9507-6470ce690c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.276896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b46b4-893f-46d9-9507-6470ce690c4f admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.277173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b46b4-893f-46d9-9507-6470ce690c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.277446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3b46b4-893f-46d9-9507-6470ce690c4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.278071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b46b4-893f-46d9-9507-6470ce690c4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.280095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3b46b4-893f-46d9-9507-6470ce690c4f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.300385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 785/1569] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:27.301322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6188e3-0c37-47a1-86f6-3c49ab1960d6 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.310964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 785/1570] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/projects => generated 375 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.310964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b46b4-893f-46d9-9507-6470ce690c4f admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.311852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6188e3-0c37-47a1-86f6-3c49ab1960d6 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.369721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a452611-cd08-4737-b69a-0b9b0e38bb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.370399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a452611-cd08-4737-b69a-0b9b0e38bb2b admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.370517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a452611-cd08-4737-b69a-0b9b0e38bb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.370785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a452611-cd08-4737-b69a-0b9b0e38bb2b admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.371403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a452611-cd08-4737-b69a-0b9b0e38bb2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.373893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a452611-cd08-4737-b69a-0b9b0e38bb2b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.374585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3b46b4-893f-46d9-9507-6470ce690c4f admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.379278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6697b-c409-4dbf-ba73-344d6784c867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.379904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6697b-c409-4dbf-ba73-344d6784c867 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.380176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6697b-c409-4dbf-ba73-344d6784c867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.380440 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad6697b-c409-4dbf-ba73-344d6784c867 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.381104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6697b-c409-4dbf-ba73-344d6784c867 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:27.381401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6697b-c409-4dbf-ba73-344d6784c867 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.382974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad6697b-c409-4dbf-ba73-344d6784c867 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.395988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 786/1571] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:27.399841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 786/1572] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.399991 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6697b-c409-4dbf-ba73-344d6784c867 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.400110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a452611-cd08-4737-b69a-0b9b0e38bb2b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.408692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a452611-cd08-4737-b69a-0b9b0e38bb2b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.408973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad6697b-c409-4dbf-ba73-344d6784c867 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.413268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfb770-79bb-4b06-88e2-6027db459b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.413769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ed90c-8a8c-4836-9fa4-8f8d7e0390c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.413903 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfb770-79bb-4b06-88e2-6027db459b86 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.414123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfb770-79bb-4b06-88e2-6027db459b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.414356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ed90c-8a8c-4836-9fa4-8f8d7e0390c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.414472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dfb770-79bb-4b06-88e2-6027db459b86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.414656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ed90c-8a8c-4836-9fa4-8f8d7e0390c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.414922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550ed90c-8a8c-4836-9fa4-8f8d7e0390c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.415016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dfb770-79bb-4b06-88e2-6027db459b86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.415633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ed90c-8a8c-4836-9fa4-8f8d7e0390c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:27.415932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ed90c-8a8c-4836-9fa4-8f8d7e0390c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.417498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dfb770-79bb-4b06-88e2-6027db459b86 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.417625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550ed90c-8a8c-4836-9fa4-8f8d7e0390c8 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.433040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 787/1573] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:27.433647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ed90c-8a8c-4836-9fa4-8f8d7e0390c8 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.443193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550ed90c-8a8c-4836-9fa4-8f8d7e0390c8 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.448239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ffdfd-49ec-40b3-b312-1a5518494cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.448970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ffdfd-49ec-40b3-b312-1a5518494cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.449302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ffdfd-49ec-40b3-b312-1a5518494cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.449572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476ffdfd-49ec-40b3-b312-1a5518494cd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.450205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ffdfd-49ec-40b3-b312-1a5518494cd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.452031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: WARNING py.warnings [None req-476ffdfd-49ec-40b3-b312-1a5518494cd5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:13:27.452031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: warnings.warn(msg) Sep 10 14:13:27.452031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: Sep 10 14:13:27.453233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476ffdfd-49ec-40b3-b312-1a5518494cd5 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.458247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 787/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/projects => generated 357 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.459333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfb770-79bb-4b06-88e2-6027db459b86 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.472780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:27.474845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 788/1575] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/users => generated 289 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.475690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ffdfd-49ec-40b3-b312-1a5518494cd5 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.486647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476ffdfd-49ec-40b3-b312-1a5518494cd5 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.495472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:27.527992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dfb770-79bb-4b06-88e2-6027db459b86 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.533655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b086ae-5d9a-4e09-b722-bb9b92db9a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.534255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b086ae-5d9a-4e09-b722-bb9b92db9a7c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.534527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b086ae-5d9a-4e09-b722-bb9b92db9a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.534794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b086ae-5d9a-4e09-b722-bb9b92db9a7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.535414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b086ae-5d9a-4e09-b722-bb9b92db9a7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.537366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b086ae-5d9a-4e09-b722-bb9b92db9a7c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.545349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebfdf9-e523-49ad-bf68-5f8ce4d053f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.546276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebfdf9-e523-49ad-bf68-5f8ce4d053f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.546561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebfdf9-e523-49ad-bf68-5f8ce4d053f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.546830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ebfdf9-e523-49ad-bf68-5f8ce4d053f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.547483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebfdf9-e523-49ad-bf68-5f8ce4d053f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.549389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ebfdf9-e523-49ad-bf68-5f8ce4d053f2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.565744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 788/1576] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/projects => generated 391 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.566599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b086ae-5d9a-4e09-b722-bb9b92db9a7c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.574071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 789/1577] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.577196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebfdf9-e523-49ad-bf68-5f8ce4d053f2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.586933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ebfdf9-e523-49ad-bf68-5f8ce4d053f2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.621524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b086ae-5d9a-4e09-b722-bb9b92db9a7c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.625179 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54525d74-ca51-4412-8300-aaf9dc50641b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.625803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525d74-ca51-4412-8300-aaf9dc50641b service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.626074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525d74-ca51-4412-8300-aaf9dc50641b service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.626340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54525d74-ca51-4412-8300-aaf9dc50641b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.653051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccafac-b9a9-491d-861b-1b5b4bd11416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.653785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccafac-b9a9-491d-861b-1b5b4bd11416 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.654587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccafac-b9a9-491d-861b-1b5b4bd11416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.655037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ccafac-b9a9-491d-861b-1b5b4bd11416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.655955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccafac-b9a9-491d-861b-1b5b4bd11416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.657840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ccafac-b9a9-491d-861b-1b5b4bd11416 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.667771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 790/1578] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:27.667771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccafac-b9a9-491d-861b-1b5b4bd11416 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.679123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ccafac-b9a9-491d-861b-1b5b4bd11416 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.684124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858f179-8379-40d8-ac98-ece76b9cad33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.684705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858f179-8379-40d8-ac98-ece76b9cad33 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.684980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858f179-8379-40d8-ac98-ece76b9cad33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.685269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c858f179-8379-40d8-ac98-ece76b9cad33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.686224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858f179-8379-40d8-ac98-ece76b9cad33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.688180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c858f179-8379-40d8-ac98-ece76b9cad33 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.690024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54525d74-ca51-4412-8300-aaf9dc50641b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.691497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54525d74-ca51-4412-8300-aaf9dc50641b service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.700891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 789/1579] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:27.701671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54525d74-ca51-4412-8300-aaf9dc50641b service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.711094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54525d74-ca51-4412-8300-aaf9dc50641b service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.716061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b6821-145d-48a9-b361-7d772a021ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.717205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b6821-145d-48a9-b361-7d772a021ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.717483 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b6821-145d-48a9-b361-7d772a021ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.717751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b6821-145d-48a9-b361-7d772a021ef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.718495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b6821-145d-48a9-b361-7d772a021ef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.719617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 791/1580] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/projects => generated 389 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.720458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b6821-145d-48a9-b361-7d772a021ef5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.720992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858f179-8379-40d8-ac98-ece76b9cad33 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.750142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 790/1581] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.754154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b6821-145d-48a9-b361-7d772a021ef5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.784317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:27.794931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c858f179-8379-40d8-ac98-ece76b9cad33 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.819009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:27.835680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b6821-145d-48a9-b361-7d772a021ef5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.841568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba387915-d5f4-4dba-91a1-cd01830ef578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.842914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba387915-d5f4-4dba-91a1-cd01830ef578 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.843263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba387915-d5f4-4dba-91a1-cd01830ef578 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.843645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba387915-d5f4-4dba-91a1-cd01830ef578 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.844382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba387915-d5f4-4dba-91a1-cd01830ef578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.846201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba387915-d5f4-4dba-91a1-cd01830ef578 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.857783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 791/1582] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:27.860730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba387915-d5f4-4dba-91a1-cd01830ef578 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.862011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa7888-795a-4f9d-8472-d20553db0b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.862662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa7888-795a-4f9d-8472-d20553db0b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.862944 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa7888-795a-4f9d-8472-d20553db0b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.863221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fa7888-795a-4f9d-8472-d20553db0b54 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.863913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa7888-795a-4f9d-8472-d20553db0b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.865865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fa7888-795a-4f9d-8472-d20553db0b54 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.874797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba387915-d5f4-4dba-91a1-cd01830ef578 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.881527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea3648-1f56-42bd-946c-0812071cd846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.882355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea3648-1f56-42bd-946c-0812071cd846 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.882459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea3648-1f56-42bd-946c-0812071cd846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.882781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea3648-1f56-42bd-946c-0812071cd846 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.883634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea3648-1f56-42bd-946c-0812071cd846 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.887705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ea3648-1f56-42bd-946c-0812071cd846 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.891850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 792/1583] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/users => generated 279 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.895863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa7888-795a-4f9d-8472-d20553db0b54 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.908981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fa7888-795a-4f9d-8472-d20553db0b54 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.915403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67b877-7d74-437d-b908-f1d3fc3b5939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.916131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67b877-7d74-437d-b908-f1d3fc3b5939 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.916400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67b877-7d74-437d-b908-f1d3fc3b5939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.916713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d67b877-7d74-437d-b908-f1d3fc3b5939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.917348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67b877-7d74-437d-b908-f1d3fc3b5939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.919115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d67b877-7d74-437d-b908-f1d3fc3b5939 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.925598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 792/1584] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/users => generated 296 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.926528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea3648-1f56-42bd-946c-0812071cd846 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.935667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 793/1585] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:27.935667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67b877-7d74-437d-b908-f1d3fc3b5939 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.937206 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea3648-1f56-42bd-946c-0812071cd846 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.943229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa8550-1e71-437c-a5a5-15fd61c5b694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.944045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa8550-1e71-437c-a5a5-15fd61c5b694 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.944423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa8550-1e71-437c-a5a5-15fd61c5b694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.944827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa8550-1e71-437c-a5a5-15fd61c5b694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.945573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa8550-1e71-437c-a5a5-15fd61c5b694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.946341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d67b877-7d74-437d-b908-f1d3fc3b5939 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.947378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa8550-1e71-437c-a5a5-15fd61c5b694 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.952603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067caf3-1249-47e5-bad8-90c6cf84983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.953210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067caf3-1249-47e5-bad8-90c6cf84983c admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.953486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067caf3-1249-47e5-bad8-90c6cf84983c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.953811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4067caf3-1249-47e5-bad8-90c6cf84983c admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.954445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067caf3-1249-47e5-bad8-90c6cf84983c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.956880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4067caf3-1249-47e5-bad8-90c6cf84983c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.961938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 793/1586] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:27.963435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa8550-1e71-437c-a5a5-15fd61c5b694 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.978171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa8550-1e71-437c-a5a5-15fd61c5b694 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:27.986319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 794/1587] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/users => generated 295 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:27.986319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067caf3-1249-47e5-bad8-90c6cf84983c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:27.987691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12b3af-7a26-49fd-a7f2-7e48665dcfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:27.988372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12b3af-7a26-49fd-a7f2-7e48665dcfdb admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:27.988753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12b3af-7a26-49fd-a7f2-7e48665dcfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:27.989099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f12b3af-7a26-49fd-a7f2-7e48665dcfdb admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:27.989805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12b3af-7a26-49fd-a7f2-7e48665dcfdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:27.992172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f12b3af-7a26-49fd-a7f2-7e48665dcfdb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:27.996804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4067caf3-1249-47e5-bad8-90c6cf84983c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.001735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c0c86-9798-40de-b7e3-b6448860d74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.002858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c0c86-9798-40de-b7e3-b6448860d74c admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.003210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c0c86-9798-40de-b7e3-b6448860d74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.003890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18c0c86-9798-40de-b7e3-b6448860d74c admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.004873 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c0c86-9798-40de-b7e3-b6448860d74c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.006668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18c0c86-9798-40de-b7e3-b6448860d74c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.015786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 794/1588] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:27 2022] POST /identity/v3/users => generated 303 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:28.017406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12b3af-7a26-49fd-a7f2-7e48665dcfdb admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.020951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 795/1589] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:27 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:28.022914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c0c86-9798-40de-b7e3-b6448860d74c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.034282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f12b3af-7a26-49fd-a7f2-7e48665dcfdb admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.035227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18c0c86-9798-40de-b7e3-b6448860d74c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.039254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba400ab-b795-46c2-a52d-c56809cc77ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.040044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba400ab-b795-46c2-a52d-c56809cc77ff admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.040185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce472b1a-5004-448a-8c34-95fa90621150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.040515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba400ab-b795-46c2-a52d-c56809cc77ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.040786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472b1a-5004-448a-8c34-95fa90621150 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.040980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba400ab-b795-46c2-a52d-c56809cc77ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.041131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472b1a-5004-448a-8c34-95fa90621150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.041385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472b1a-5004-448a-8c34-95fa90621150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.041891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba400ab-b795-46c2-a52d-c56809cc77ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.042017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce472b1a-5004-448a-8c34-95fa90621150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.043673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce472b1a-5004-448a-8c34-95fa90621150 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.043951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba400ab-b795-46c2-a52d-c56809cc77ff admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.054152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 796/1590] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:28.054834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce472b1a-5004-448a-8c34-95fa90621150 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.057215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 795/1591] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:28.057981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba400ab-b795-46c2-a52d-c56809cc77ff admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.065072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce472b1a-5004-448a-8c34-95fa90621150 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.070328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94c9b1-2d2b-4ef0-8e9b-2e998cb4c63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.070912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94c9b1-2d2b-4ef0-8e9b-2e998cb4c63d admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.071262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94c9b1-2d2b-4ef0-8e9b-2e998cb4c63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.071563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94c9b1-2d2b-4ef0-8e9b-2e998cb4c63d admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.072346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94c9b1-2d2b-4ef0-8e9b-2e998cb4c63d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.074105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94c9b1-2d2b-4ef0-8e9b-2e998cb4c63d admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.076889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba400ab-b795-46c2-a52d-c56809cc77ff admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.082568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab052c5a-5429-4743-8117-b48d4cb91e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.083406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab052c5a-5429-4743-8117-b48d4cb91e31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.083881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab052c5a-5429-4743-8117-b48d4cb91e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.084351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab052c5a-5429-4743-8117-b48d4cb91e31 admin admin] PATH_INFO: `/v3/projects/06101fb7de2e4338967b29365e713677/users/5ca58aca3f9d4d8fa9f1c6315db7f54d/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.090213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 797/1592] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:28.090213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94c9b1-2d2b-4ef0-8e9b-2e998cb4c63d admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.102699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94c9b1-2d2b-4ef0-8e9b-2e998cb4c63d admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.108263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64cf12-58da-4e65-be36-8492df663198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.108869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64cf12-58da-4e65-be36-8492df663198 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.109146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64cf12-58da-4e65-be36-8492df663198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.109452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb64cf12-58da-4e65-be36-8492df663198 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.110052 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64cf12-58da-4e65-be36-8492df663198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.111161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab052c5a-5429-4743-8117-b48d4cb91e31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06101fb7de2e4338967b29365e713677, user_id=5ca58aca3f9d4d8fa9f1c6315db7f54d, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.111721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb64cf12-58da-4e65-be36-8492df663198 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.114865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab052c5a-5429-4743-8117-b48d4cb91e31 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.123874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 798/1593] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:28.125241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64cf12-58da-4e65-be36-8492df663198 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.134573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 796/1594] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:28 2022] PUT /identity/v3/projects/06101fb7de2e4338967b29365e713677/users/5ca58aca3f9d4d8fa9f1c6315db7f54d/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:28.136200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab052c5a-5429-4743-8117-b48d4cb91e31 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.140866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb64cf12-58da-4e65-be36-8492df663198 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.204186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab052c5a-5429-4743-8117-b48d4cb91e31 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.209127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3501fac-1b50-4ac7-9f0e-74a666f9d80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.209929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3501fac-1b50-4ac7-9f0e-74a666f9d80c admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.210314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3501fac-1b50-4ac7-9f0e-74a666f9d80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.210689 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3501fac-1b50-4ac7-9f0e-74a666f9d80c admin admin] PATH_INFO: `/v3/projects/6efda1ce3d4d4dbbb5e8d5005902924d/users/8ea8ba61dfa54cbc9a45543d20f31e76/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.216113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ba8dd-ecd1-4327-872d-0b75d1f85eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.216780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ba8dd-ecd1-4327-872d-0b75d1f85eca admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.217058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ba8dd-ecd1-4327-872d-0b75d1f85eca admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.217326 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89ba8dd-ecd1-4327-872d-0b75d1f85eca admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.217992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ba8dd-ecd1-4327-872d-0b75d1f85eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.220537 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89ba8dd-ecd1-4327-872d-0b75d1f85eca admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.227220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3501fac-1b50-4ac7-9f0e-74a666f9d80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6efda1ce3d4d4dbbb5e8d5005902924d, user_id=8ea8ba61dfa54cbc9a45543d20f31e76, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.230196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3501fac-1b50-4ac7-9f0e-74a666f9d80c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.234989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 799/1595] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:28.234989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ba8dd-ecd1-4327-872d-0b75d1f85eca admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.246044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 797/1596] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:28 2022] PUT /identity/v3/projects/6efda1ce3d4d4dbbb5e8d5005902924d/users/8ea8ba61dfa54cbc9a45543d20f31e76/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:28.246825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89ba8dd-ecd1-4327-872d-0b75d1f85eca admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.247386 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3501fac-1b50-4ac7-9f0e-74a666f9d80c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.272110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:28.290699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:28.319542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e657e14-9da6-4fc2-814f-bc228d453d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.320902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e657e14-9da6-4fc2-814f-bc228d453d88 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.321187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e657e14-9da6-4fc2-814f-bc228d453d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.321462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e657e14-9da6-4fc2-814f-bc228d453d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.322107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e657e14-9da6-4fc2-814f-bc228d453d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.323774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e657e14-9da6-4fc2-814f-bc228d453d88 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.325612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3501fac-1b50-4ac7-9f0e-74a666f9d80c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.331836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373eff53-0819-4add-bc54-2df02ca41375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.332848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eff53-0819-4add-bc54-2df02ca41375 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.333123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eff53-0819-4add-bc54-2df02ca41375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.333416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373eff53-0819-4add-bc54-2df02ca41375 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.334146 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373eff53-0819-4add-bc54-2df02ca41375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.335856 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373eff53-0819-4add-bc54-2df02ca41375 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.339968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 800/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:28.339968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e657e14-9da6-4fc2-814f-bc228d453d88 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.351413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 798/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:28.351413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373eff53-0819-4add-bc54-2df02ca41375 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.351704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e657e14-9da6-4fc2-814f-bc228d453d88 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.356320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b094d-ade6-4679-81f6-5b22de6151a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.357163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b094d-ade6-4679-81f6-5b22de6151a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.357557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b094d-ade6-4679-81f6-5b22de6151a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.358026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b094d-ade6-4679-81f6-5b22de6151a4 admin admin] PATH_INFO: `/v3/projects/518d6393ac634ebdaf81fa58432e0422/users/4750e4e76e46440b887c7d878812916d/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.362472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373eff53-0819-4add-bc54-2df02ca41375 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.367057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5195164-abff-4cd1-818e-04bdbe4da286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.367696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5195164-abff-4cd1-818e-04bdbe4da286 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.367973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5195164-abff-4cd1-818e-04bdbe4da286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.368247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5195164-abff-4cd1-818e-04bdbe4da286 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.368883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5195164-abff-4cd1-818e-04bdbe4da286 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.370507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5195164-abff-4cd1-818e-04bdbe4da286 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.378464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b094d-ade6-4679-81f6-5b22de6151a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518d6393ac634ebdaf81fa58432e0422, user_id=4750e4e76e46440b887c7d878812916d, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.381428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 799/1599] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:28.382004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b094d-ade6-4679-81f6-5b22de6151a4 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.383061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ecc13-7398-41db-8ba2-071bd811a45e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.383345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ecc13-7398-41db-8ba2-071bd811a45e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.383818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ecc13-7398-41db-8ba2-071bd811a45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.397388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 801/1600] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:28 2022] PUT /identity/v3/projects/518d6393ac634ebdaf81fa58432e0422/users/4750e4e76e46440b887c7d878812916d/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:28.398220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b094d-ade6-4679-81f6-5b22de6151a4 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.405814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-750ecc13-7398-41db-8ba2-071bd811a45e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:28.407027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-750ecc13-7398-41db-8ba2-071bd811a45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:28.483342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 800/1601] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:28.484909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8f8cb-ec1e-4cb7-8e78-bc15d2e87142 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.485118 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8f8cb-ec1e-4cb7-8e78-bc15d2e87142 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.485397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8f8cb-ec1e-4cb7-8e78-bc15d2e87142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.494430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b094d-ade6-4679-81f6-5b22de6151a4 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.500993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd3012-aaec-4cbb-8c95-b2869145c584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.502104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd3012-aaec-4cbb-8c95-b2869145c584 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.502652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd3012-aaec-4cbb-8c95-b2869145c584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.503129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd3012-aaec-4cbb-8c95-b2869145c584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.504590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd3012-aaec-4cbb-8c95-b2869145c584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.506954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd3012-aaec-4cbb-8c95-b2869145c584 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.513968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d5a8f8cb-ec1e-4cb7-8e78-bc15d2e87142 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:28.515636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d5a8f8cb-ec1e-4cb7-8e78-bc15d2e87142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:28.519122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 802/1602] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:28.520173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd3012-aaec-4cbb-8c95-b2869145c584 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.525776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 801/1603] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:28.529106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8f8cb-ec1e-4cb7-8e78-bc15d2e87142 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.534431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd3012-aaec-4cbb-8c95-b2869145c584 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.539723 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8f8cb-ec1e-4cb7-8e78-bc15d2e87142 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.540717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de7a483-40c9-41a1-bf71-df8d6284eaec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.541622 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7a483-40c9-41a1-bf71-df8d6284eaec admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.542045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7a483-40c9-41a1-bf71-df8d6284eaec admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.542488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de7a483-40c9-41a1-bf71-df8d6284eaec admin admin] PATH_INFO: `/v3/projects/cffd972dec364c89bbc04fd078dc7fd9/users/8f96c9a2a49047b5b8abd3b2c1475a18/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.546194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd5763-eb04-4e7b-ae0f-c35c08f759b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.547191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd5763-eb04-4e7b-ae0f-c35c08f759b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.547692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd5763-eb04-4e7b-ae0f-c35c08f759b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.548188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fd5763-eb04-4e7b-ae0f-c35c08f759b5 admin admin] PATH_INFO: `/v3/projects/81a7db658c9f4e9babeab02e89b18946/users/c27c43059bf74810b9953ac43ba3d72a/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.571778 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd5763-eb04-4e7b-ae0f-c35c08f759b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81a7db658c9f4e9babeab02e89b18946, user_id=c27c43059bf74810b9953ac43ba3d72a, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.574753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fd5763-eb04-4e7b-ae0f-c35c08f759b5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.581633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7a483-40c9-41a1-bf71-df8d6284eaec admin admin] RBAC: Authorizing `identity:create_grant(project_id=cffd972dec364c89bbc04fd078dc7fd9, user_id=8f96c9a2a49047b5b8abd3b2c1475a18, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.585600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de7a483-40c9-41a1-bf71-df8d6284eaec admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.594947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 802/1604] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:28 2022] PUT /identity/v3/projects/81a7db658c9f4e9babeab02e89b18946/users/c27c43059bf74810b9953ac43ba3d72a/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:28.596259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd5763-eb04-4e7b-ae0f-c35c08f759b5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.603925 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 803/1605] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:28 2022] PUT /identity/v3/projects/cffd972dec364c89bbc04fd078dc7fd9/users/8f96c9a2a49047b5b8abd3b2c1475a18/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:28.605172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad985321-248b-4458-beb0-91e050d59594 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.605803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad985321-248b-4458-beb0-91e050d59594 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.606147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad985321-248b-4458-beb0-91e050d59594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.645051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-ad985321-248b-4458-beb0-91e050d59594 None None] MFA Rules not processed for user `4750e4e76e46440b887c7d878812916d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:28.646235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ad985321-248b-4458-beb0-91e050d59594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:28.705212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fd5763-eb04-4e7b-ae0f-c35c08f759b5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.727112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:28.739187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 804/1606] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:28.739187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad985321-248b-4458-beb0-91e050d59594 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.752398 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:28.808783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6afc67-b02f-49c4-bb82-f3e2ef86c36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.809776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6afc67-b02f-49c4-bb82-f3e2ef86c36c admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.810822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6afc67-b02f-49c4-bb82-f3e2ef86c36c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.811235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6afc67-b02f-49c4-bb82-f3e2ef86c36c admin admin] PATH_INFO: `/v3/projects/7d5947e3bd184558bc1a8febcedce00f/users/810f0fa1f7774b37a09a91964c40b823/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.823997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad985321-248b-4458-beb0-91e050d59594 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.828634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6afc67-b02f-49c4-bb82-f3e2ef86c36c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d5947e3bd184558bc1a8febcedce00f, user_id=810f0fa1f7774b37a09a91964c40b823, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.829631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686d96d-9dde-40c9-9002-adf9868b6fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.829780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d96d-9dde-40c9-9002-adf9868b6fac service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.830169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d96d-9dde-40c9-9002-adf9868b6fac service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.830523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d96d-9dde-40c9-9002-adf9868b6fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.831405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6afc67-b02f-49c4-bb82-f3e2ef86c36c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.844380 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686d96d-9dde-40c9-9002-adf9868b6fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.846849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686d96d-9dde-40c9-9002-adf9868b6fac service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.852012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 803/1607] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:28 2022] PUT /identity/v3/projects/7d5947e3bd184558bc1a8febcedce00f/users/810f0fa1f7774b37a09a91964c40b823/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:28.856820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6afc67-b02f-49c4-bb82-f3e2ef86c36c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.861454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 805/1608] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:28.862993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686d96d-9dde-40c9-9002-adf9868b6fac service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:28.874730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686d96d-9dde-40c9-9002-adf9868b6fac service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.945377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258eb75c-566d-429a-87fe-1e9b1640a9d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.946186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eb75c-566d-429a-87fe-1e9b1640a9d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.946514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eb75c-566d-429a-87fe-1e9b1640a9d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.946842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258eb75c-566d-429a-87fe-1e9b1640a9d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.955029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6afc67-b02f-49c4-bb82-f3e2ef86c36c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:28.967231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258eb75c-566d-429a-87fe-1e9b1640a9d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:28.968506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708892ae-b55f-494b-ba51-97137cd216ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:28.969206 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258eb75c-566d-429a-87fe-1e9b1640a9d1 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:28.969440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708892ae-b55f-494b-ba51-97137cd216ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:28.970868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708892ae-b55f-494b-ba51-97137cd216ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:28.971188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708892ae-b55f-494b-ba51-97137cd216ed admin admin] PATH_INFO: `/v3/projects/1f7040e4e07a41318f060710509c6e54/users/8afdfd824d604b2f8f0f56d3f2c158bf/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:28.996005 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 806/1609] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:28.999910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567e769-251d-407b-bfbe-d4ed92c62974 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.000240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567e769-251d-407b-bfbe-d4ed92c62974 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.000487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567e769-251d-407b-bfbe-d4ed92c62974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.014877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708892ae-b55f-494b-ba51-97137cd216ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f7040e4e07a41318f060710509c6e54, user_id=8afdfd824d604b2f8f0f56d3f2c158bf, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:29.024403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708892ae-b55f-494b-ba51-97137cd216ed admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:29.046746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 804/1610] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:28 2022] PUT /identity/v3/projects/1f7040e4e07a41318f060710509c6e54/users/8afdfd824d604b2f8f0f56d3f2c158bf/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:29.048551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a8491-368d-4269-89eb-b63466277915 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.048859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a8491-368d-4269-89eb-b63466277915 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.049135 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88a8491-368d-4269-89eb-b63466277915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.050962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e567e769-251d-407b-bfbe-d4ed92c62974 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:29.052250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e567e769-251d-407b-bfbe-d4ed92c62974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:29.091131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b88a8491-368d-4269-89eb-b63466277915 None None] MFA Rules not processed for user `c27c43059bf74810b9953ac43ba3d72a`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:29.093474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b88a8491-368d-4269-89eb-b63466277915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:29.146157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 807/1611] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:29.149694 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c0f32-a29c-4ca3-8a64-5a56175dda07 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.150112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c0f32-a29c-4ca3-8a64-5a56175dda07 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.150770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9c0f32-a29c-4ca3-8a64-5a56175dda07 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.153281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 808/1612] 10.222.0.26 () {54 vars in 962 bytes} [Sat Sep 10 14:13:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:13:29.154497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2fc1a-0bea-4687-ba92-4e887e9b3ab5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.154782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2fc1a-0bea-4687-ba92-4e887e9b3ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.155054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f2fc1a-0bea-4687-ba92-4e887e9b3ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.181841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c6f2fc1a-0bea-4687-ba92-4e887e9b3ab5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:29.184532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c6f2fc1a-0bea-4687-ba92-4e887e9b3ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:29.199859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 809/1613] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:29.199859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99201624-c567-45eb-b572-1a23bce2b12c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.199859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99201624-c567-45eb-b572-1a23bce2b12c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.199859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99201624-c567-45eb-b572-1a23bce2b12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.219059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 805/1614] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:29 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:29.220686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a23b1-64f1-4f91-b2da-a0e4ae2ce0d4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.220980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a23b1-64f1-4f91-b2da-a0e4ae2ce0d4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.221926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a23b1-64f1-4f91-b2da-a0e4ae2ce0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.226221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-99201624-c567-45eb-b572-1a23bce2b12c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:29.227744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-99201624-c567-45eb-b572-1a23bce2b12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:29.238114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 810/1615] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:29.241708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99201624-c567-45eb-b572-1a23bce2b12c None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:29.271211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:29.287555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-235a23b1-64f1-4f91-b2da-a0e4ae2ce0d4 None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:29.289020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-235a23b1-64f1-4f91-b2da-a0e4ae2ce0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:29.303106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:29.322709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99201624-c567-45eb-b572-1a23bce2b12c None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:29.330597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e629f0-aabd-4f26-995d-f739659bb53c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:29.331767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e629f0-aabd-4f26-995d-f739659bb53c service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.332535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e629f0-aabd-4f26-995d-f739659bb53c service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.332535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e629f0-aabd-4f26-995d-f739659bb53c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.362104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e629f0-aabd-4f26-995d-f739659bb53c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:29.367064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e629f0-aabd-4f26-995d-f739659bb53c service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:29.394453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 811/1616] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:29.396790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e629f0-aabd-4f26-995d-f739659bb53c service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:29.407125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 806/1617] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Sep 10 14:13:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:29.408039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a23b1-64f1-4f91-b2da-a0e4ae2ce0d4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:29.417243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e629f0-aabd-4f26-995d-f739659bb53c service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:29.438788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a23b1-64f1-4f91-b2da-a0e4ae2ce0d4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:29.440936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d4c2e-388e-4f35-ab32-67500e800fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:29.442586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d4c2e-388e-4f35-ab32-67500e800fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.442881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d4c2e-388e-4f35-ab32-67500e800fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.443160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d4c2e-388e-4f35-ab32-67500e800fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.443844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d4c2e-388e-4f35-ab32-67500e800fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:29.445801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d4c2e-388e-4f35-ab32-67500e800fd2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:29.450353 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613b5ff-ae6c-4e0d-842d-4086fd7a4c47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:29.451705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613b5ff-ae6c-4e0d-842d-4086fd7a4c47 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.452086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613b5ff-ae6c-4e0d-842d-4086fd7a4c47 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.452365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613b5ff-ae6c-4e0d-842d-4086fd7a4c47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.463844 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613b5ff-ae6c-4e0d-842d-4086fd7a4c47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:29.465532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613b5ff-ae6c-4e0d-842d-4086fd7a4c47 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:29.479586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 807/1618] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:29.480709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613b5ff-ae6c-4e0d-842d-4086fd7a4c47 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:29.491270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613b5ff-ae6c-4e0d-842d-4086fd7a4c47 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:29.496090 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788ca90-2480-4799-b8e5-1708ea595a2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:29.496838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788ca90-2480-4799-b8e5-1708ea595a2d service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.497125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788ca90-2480-4799-b8e5-1708ea595a2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.497481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788ca90-2480-4799-b8e5-1708ea595a2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.511039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788ca90-2480-4799-b8e5-1708ea595a2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:29.511588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 812/1619] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:29 2022] POST /identity/v3/projects => generated 355 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:29.512793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7788ca90-2480-4799-b8e5-1708ea595a2d service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:29.516839 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d4c2e-388e-4f35-ab32-67500e800fd2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:29.525032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 808/1620] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:29.525942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788ca90-2480-4799-b8e5-1708ea595a2d service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:29.542169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7788ca90-2480-4799-b8e5-1708ea595a2d service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:29.630052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded8537-39bb-45d2-af4c-c712fa6e2cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:29.632429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded8537-39bb-45d2-af4c-c712fa6e2cbb admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.632980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded8537-39bb-45d2-af4c-c712fa6e2cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.633573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ded8537-39bb-45d2-af4c-c712fa6e2cbb admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.634341 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded8537-39bb-45d2-af4c-c712fa6e2cbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:29.637634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ded8537-39bb-45d2-af4c-c712fa6e2cbb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:29.672647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d4c2e-388e-4f35-ab32-67500e800fd2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:29.672891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 809/1621] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:29 2022] POST /identity/v3/users => generated 284 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:29.676971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded8537-39bb-45d2-af4c-c712fa6e2cbb admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:29.679193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a38a5-bf8f-4eed-9128-fb97d30de377 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:29.680068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a38a5-bf8f-4eed-9128-fb97d30de377 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.680521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a38a5-bf8f-4eed-9128-fb97d30de377 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.681014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a38a5-bf8f-4eed-9128-fb97d30de377 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.694892 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ded8537-39bb-45d2-af4c-c712fa6e2cbb admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:29.703123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a4901-2031-4802-b901-c7cd19d46764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:29.704014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a4901-2031-4802-b901-c7cd19d46764 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.704370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a4901-2031-4802-b901-c7cd19d46764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.704887 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a4901-2031-4802-b901-c7cd19d46764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.706624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a4901-2031-4802-b901-c7cd19d46764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:29.708901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219a4901-2031-4802-b901-c7cd19d46764 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:29.728704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 810/1622] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:29.733057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a4901-2031-4802-b901-c7cd19d46764 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:29.749810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219a4901-2031-4802-b901-c7cd19d46764 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:29.757505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6257f-f6dd-4fa6-94e2-1becd8512a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:29.757800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a38a5-bf8f-4eed-9128-fb97d30de377 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:29.758344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6257f-f6dd-4fa6-94e2-1becd8512a76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.758682 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6257f-f6dd-4fa6-94e2-1becd8512a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.759032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6257f-f6dd-4fa6-94e2-1becd8512a76 admin admin] PATH_INFO: `/v3/projects/d00b655c92d64645a3f46871849e4c4e/users/4ce12bd80b7444dcb6ed90d63157021e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.760870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731a38a5-bf8f-4eed-9128-fb97d30de377 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:29.787158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6257f-f6dd-4fa6-94e2-1becd8512a76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00b655c92d64645a3f46871849e4c4e, user_id=4ce12bd80b7444dcb6ed90d63157021e, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:29.790929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6257f-f6dd-4fa6-94e2-1becd8512a76 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:29.799855 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 813/1623] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:29.813460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 811/1624] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:29 2022] PUT /identity/v3/projects/d00b655c92d64645a3f46871849e4c4e/users/4ce12bd80b7444dcb6ed90d63157021e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:29.822203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a38a5-bf8f-4eed-9128-fb97d30de377 service nova] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:29.837070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a38a5-bf8f-4eed-9128-fb97d30de377 service nova] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:29.951619 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9750269-a7a5-4b6f-aedb-7ec95b7e0efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:29.954391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9750269-a7a5-4b6f-aedb-7ec95b7e0efa admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:29.954935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9750269-a7a5-4b6f-aedb-7ec95b7e0efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:29.956601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9750269-a7a5-4b6f-aedb-7ec95b7e0efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:29.958222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9750269-a7a5-4b6f-aedb-7ec95b7e0efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:29.960280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9750269-a7a5-4b6f-aedb-7ec95b7e0efa admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:29.963124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:29.979850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 814/1625] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:29.984300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6257f-f6dd-4fa6-94e2-1becd8512a76 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:30.002775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6257f-f6dd-4fa6-94e2-1becd8512a76 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:30.010847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074827f-0041-4667-8cf2-c696e5ae6645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:30.012073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074827f-0041-4667-8cf2-c696e5ae6645 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:30.012810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074827f-0041-4667-8cf2-c696e5ae6645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:30.013881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7074827f-0041-4667-8cf2-c696e5ae6645 admin admin] PATH_INFO: `/v3/projects/d00b655c92d64645a3f46871849e4c4e/users/4ce12bd80b7444dcb6ed90d63157021e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:30.016557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074827f-0041-4667-8cf2-c696e5ae6645 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d00b655c92d64645a3f46871849e4c4e, user_id=4ce12bd80b7444dcb6ed90d63157021e, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:30.020203 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7074827f-0041-4667-8cf2-c696e5ae6645 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:30.064235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 812/1626] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:29 2022] PUT /identity/v3/projects/d00b655c92d64645a3f46871849e4c4e/users/4ce12bd80b7444dcb6ed90d63157021e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:30.066029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e3841-593b-4446-91b9-4f91a0584386 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:30.066299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e3841-593b-4446-91b9-4f91a0584386 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:30.066556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4e3841-593b-4446-91b9-4f91a0584386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:30.164021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-0e4e3841-593b-4446-91b9-4f91a0584386 None None] MFA Rules not processed for user `4ce12bd80b7444dcb6ed90d63157021e`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:30.164021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0e4e3841-593b-4446-91b9-4f91a0584386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:30.289743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 815/1627] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:13:30.302734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074827f-0041-4667-8cf2-c696e5ae6645 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:30.331336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:30.415945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7074827f-0041-4667-8cf2-c696e5ae6645 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:30.422941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9313c-e5bd-4d5a-bdb9-6b06b58d7e46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:30.424404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9313c-e5bd-4d5a-bdb9-6b06b58d7e46 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:30.424988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9313c-e5bd-4d5a-bdb9-6b06b58d7e46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:30.425551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9313c-e5bd-4d5a-bdb9-6b06b58d7e46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:30.443222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9313c-e5bd-4d5a-bdb9-6b06b58d7e46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:30.448282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9313c-e5bd-4d5a-bdb9-6b06b58d7e46 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:30.478828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 813/1628] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:30.604316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e3841-593b-4446-91b9-4f91a0584386 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:30.619372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4e3841-593b-4446-91b9-4f91a0584386 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:30.620172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db25e0-a6d8-47e0-8d95-9272880fc350 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:30.620459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db25e0-a6d8-47e0-8d95-9272880fc350 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:30.620756 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44db25e0-a6d8-47e0-8d95-9272880fc350 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:30.622826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 814/1629] 10.222.0.26 () {54 vars in 962 bytes} [Sat Sep 10 14:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:13:30.625599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf342ec5-36e4-4a3d-b5b5-93459d5cc9fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:30.632948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf342ec5-36e4-4a3d-b5b5-93459d5cc9fe service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:30.634146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf342ec5-36e4-4a3d-b5b5-93459d5cc9fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:30.634430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf342ec5-36e4-4a3d-b5b5-93459d5cc9fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:30.648503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0690c26-bdfd-4614-9e3d-4f938e0e42be None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:30.648791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0690c26-bdfd-4614-9e3d-4f938e0e42be None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:30.649103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0690c26-bdfd-4614-9e3d-4f938e0e42be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:30.694818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a0690c26-bdfd-4614-9e3d-4f938e0e42be None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:30.696113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a0690c26-bdfd-4614-9e3d-4f938e0e42be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:30.745742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf342ec5-36e4-4a3d-b5b5-93459d5cc9fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:30.747266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf342ec5-36e4-4a3d-b5b5-93459d5cc9fe service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:30.766230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 816/1630] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:13:30.873295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 815/1631] 10.222.0.26 () {58 vars in 1059 bytes} [Sat Sep 10 14:13:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:13:30.879858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d3034-c203-4b97-b3e0-74aa4d1598da None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:30.880334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d3034-c203-4b97-b3e0-74aa4d1598da None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:30.880791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d3034-c203-4b97-b3e0-74aa4d1598da None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:30.882497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 817/1632] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 10 14:13:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:13:30.889088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bc320-dab4-4eb1-91ce-d1fbe58f4e5f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:30.889699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bc320-dab4-4eb1-91ce-d1fbe58f4e5f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:30.890148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5bc320-dab4-4eb1-91ce-d1fbe58f4e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:30.925540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ad5bc320-dab4-4eb1-91ce-d1fbe58f4e5f None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:30.930368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ad5bc320-dab4-4eb1-91ce-d1fbe58f4e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:30.952526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 816/1633] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:13:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:30.971102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d3034-c203-4b97-b3e0-74aa4d1598da None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:31.006760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d3034-c203-4b97-b3e0-74aa4d1598da None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:31.011654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e59a14-11ee-4335-894d-7f8983d22e7a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:31.013232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e59a14-11ee-4335-894d-7f8983d22e7a service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:31.013874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e59a14-11ee-4335-894d-7f8983d22e7a service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:31.014682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e59a14-11ee-4335-894d-7f8983d22e7a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:31.048933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e59a14-11ee-4335-894d-7f8983d22e7a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:31.050711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e59a14-11ee-4335-894d-7f8983d22e7a service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:31.071633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 818/1634] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:31.266832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baae60-a1dd-4cf9-b957-61382611abda None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:31.267182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baae60-a1dd-4cf9-b957-61382611abda None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:31.267557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8baae60-a1dd-4cf9-b957-61382611abda None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:31.270347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 817/1635] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 10 14:13:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:13:31.276749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462a03f-d4ed-4089-88d0-de1efbe8bc75 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:31.277313 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462a03f-d4ed-4089-88d0-de1efbe8bc75 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:31.277713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462a03f-d4ed-4089-88d0-de1efbe8bc75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:31.304208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-9462a03f-d4ed-4089-88d0-de1efbe8bc75 None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:31.305664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9462a03f-d4ed-4089-88d0-de1efbe8bc75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:31.319681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 819/1636] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:13:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:31.343370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baae60-a1dd-4cf9-b957-61382611abda None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:31.372170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8baae60-a1dd-4cf9-b957-61382611abda None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:31.385552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839786a5-57ba-4f53-9e9a-255222b90f33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:31.389789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839786a5-57ba-4f53-9e9a-255222b90f33 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:31.390534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839786a5-57ba-4f53-9e9a-255222b90f33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:31.391348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839786a5-57ba-4f53-9e9a-255222b90f33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:31.407129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839786a5-57ba-4f53-9e9a-255222b90f33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:31.408632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839786a5-57ba-4f53-9e9a-255222b90f33 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:31.425760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 818/1637] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:31.442449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7df93-ee0b-4cf1-8ec4-d01f676cd296 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:31.442736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7df93-ee0b-4cf1-8ec4-d01f676cd296 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:31.444254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb7df93-ee0b-4cf1-8ec4-d01f676cd296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:31.479276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2bb7df93-ee0b-4cf1-8ec4-d01f676cd296 None None] MFA Rules not processed for user `8afdfd824d604b2f8f0f56d3f2c158bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:31.481205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2bb7df93-ee0b-4cf1-8ec4-d01f676cd296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:31.574263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:31.617561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 820/1638] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:31 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:31.621853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839786a5-57ba-4f53-9e9a-255222b90f33 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:31.660092 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7df93-ee0b-4cf1-8ec4-d01f676cd296 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:31.785376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839786a5-57ba-4f53-9e9a-255222b90f33 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:31.797278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b044d28-311c-4404-85ef-46d47d5722c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:31.800106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb7df93-ee0b-4cf1-8ec4-d01f676cd296 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:31.800349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b044d28-311c-4404-85ef-46d47d5722c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:31.800708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b044d28-311c-4404-85ef-46d47d5722c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:31.801230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b044d28-311c-4404-85ef-46d47d5722c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:31.802047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b044d28-311c-4404-85ef-46d47d5722c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:31.805056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b044d28-311c-4404-85ef-46d47d5722c9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:31.815749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:31.816417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:31.816693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:31.816962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] PATH_INFO: `/v3/users/4750e4e76e46440b887c7d878812916d` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:31.818968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4750e4e76e46440b887c7d878812916d)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:31.821236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:31.871982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 819/1639] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:31 2022] POST /identity/v3/projects => generated 405 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:31.876692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b044d28-311c-4404-85ef-46d47d5722c9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:31.906902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:31.951239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4750e4e76e46440b887c7d878812916d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:31.953241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4750e4e76e46440b887c7d878812916d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:31.971826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4750e4e76e46440b887c7d878812916d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:31.988091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4750e4e76e46440b887c7d878812916d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:31.991301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1124dad7-0cd6-4d4a-ab73-c2a5bdec745b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4750e4e76e46440b887c7d878812916d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:32.041932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 821/1640] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:13:31 2022] DELETE /identity/v3/users/4750e4e76e46440b887c7d878812916d => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:32.044734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7eca-d265-4280-bffc-2af0dca32782 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:32.044734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7eca-d265-4280-bffc-2af0dca32782 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:32.047085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee7eca-d265-4280-bffc-2af0dca32782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:32.070051 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b044d28-311c-4404-85ef-46d47d5722c9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:32.102854 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-09ee7eca-d265-4280-bffc-2af0dca32782 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:32.104463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-09ee7eca-d265-4280-bffc-2af0dca32782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:32.223198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be9ce-6b83-44d2-b374-533eece420a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:32.223767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be9ce-6b83-44d2-b374-533eece420a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:32.224044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be9ce-6b83-44d2-b374-533eece420a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:32.224315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663be9ce-6b83-44d2-b374-533eece420a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:32.225583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663be9ce-6b83-44d2-b374-533eece420a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:32.229650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663be9ce-6b83-44d2-b374-533eece420a4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:32.243958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 822/1641] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:32.270680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee7eca-d265-4280-bffc-2af0dca32782 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:32.273274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-09ee7eca-d265-4280-bffc-2af0dca32782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:32.280059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 820/1642] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:31 2022] POST /identity/v3/users => generated 309 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:32.283402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be9ce-6b83-44d2-b374-533eece420a4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:32.286439 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-663be9ce-6b83-44d2-b374-533eece420a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:32.303458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:32.329792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663be9ce-6b83-44d2-b374-533eece420a4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:32.348410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f49f4-34a4-47f9-b51c-52689c08a13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:32.349194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f49f4-34a4-47f9-b51c-52689c08a13e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:32.349431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f49f4-34a4-47f9-b51c-52689c08a13e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:32.350283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8f49f4-34a4-47f9-b51c-52689c08a13e admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:32.352129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f49f4-34a4-47f9-b51c-52689c08a13e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:32.363312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8f49f4-34a4-47f9-b51c-52689c08a13e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:32.442683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 821/1643] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:32.444453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f49f4-34a4-47f9-b51c-52689c08a13e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:32.459547 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee7eca-d265-4280-bffc-2af0dca32782 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:32.471071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbbdd7-6533-4a94-a995-05dcbec36cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:32.471794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbbdd7-6533-4a94-a995-05dcbec36cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:32.472070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbbdd7-6533-4a94-a995-05dcbec36cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:32.472338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbbdd7-6533-4a94-a995-05dcbec36cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:32.498244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8f49f4-34a4-47f9-b51c-52689c08a13e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:32.498909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbbdd7-6533-4a94-a995-05dcbec36cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:32.498909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fbbdd7-6533-4a94-a995-05dcbec36cd9 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:32.508159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77c8d0-c5f8-44b2-a736-dd1632944041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:32.509772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77c8d0-c5f8-44b2-a736-dd1632944041 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:32.510694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77c8d0-c5f8-44b2-a736-dd1632944041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:32.511316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77c8d0-c5f8-44b2-a736-dd1632944041 admin admin] PATH_INFO: `/v3/projects/d4b16a80e9c0448a9e8382c229390678/users/f7d4fc9ff073439a94b62638d33381e5/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:32.520974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 823/1644] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:32.534492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77c8d0-c5f8-44b2-a736-dd1632944041 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b16a80e9c0448a9e8382c229390678, user_id=f7d4fc9ff073439a94b62638d33381e5, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:32.539941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77c8d0-c5f8-44b2-a736-dd1632944041 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:32.558914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 822/1645] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:32 2022] PUT /identity/v3/projects/d4b16a80e9c0448a9e8382c229390678/users/f7d4fc9ff073439a94b62638d33381e5/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:32.563719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbbdd7-6533-4a94-a995-05dcbec36cd9 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:32.579639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fbbdd7-6533-4a94-a995-05dcbec36cd9 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:32.739018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070d124-5287-4626-bf80-a34238fde1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:32.740432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070d124-5287-4626-bf80-a34238fde1f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:32.740904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070d124-5287-4626-bf80-a34238fde1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:32.741365 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070d124-5287-4626-bf80-a34238fde1f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:32.749144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070d124-5287-4626-bf80-a34238fde1f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:32.751412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4070d124-5287-4626-bf80-a34238fde1f1 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:32.769224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77c8d0-c5f8-44b2-a736-dd1632944041 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:32.770221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 824/1646] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:32.783261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77c8d0-c5f8-44b2-a736-dd1632944041 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:32.794102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131dc73-1f0c-4412-aa6e-a0933041801b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:32.795205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131dc73-1f0c-4412-aa6e-a0933041801b admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:32.795731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131dc73-1f0c-4412-aa6e-a0933041801b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:32.796200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131dc73-1f0c-4412-aa6e-a0933041801b admin admin] PATH_INFO: `/v3/projects/d4b16a80e9c0448a9e8382c229390678/users/f7d4fc9ff073439a94b62638d33381e5/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:32.802471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131dc73-1f0c-4412-aa6e-a0933041801b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b16a80e9c0448a9e8382c229390678, user_id=f7d4fc9ff073439a94b62638d33381e5, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:32.806180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131dc73-1f0c-4412-aa6e-a0933041801b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:32.842850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 823/1647] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:32 2022] PUT /identity/v3/projects/d4b16a80e9c0448a9e8382c229390678/users/f7d4fc9ff073439a94b62638d33381e5/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:33.762888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070d124-5287-4626-bf80-a34238fde1f1 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:33.765492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4070d124-5287-4626-bf80-a34238fde1f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:33.786013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070d124-5287-4626-bf80-a34238fde1f1 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:33.940921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:33.941885 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:33.944189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:33.945246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] PATH_INFO: `/v3/projects/518d6393ac634ebdaf81fa58432e0422` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:33.956137 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=518d6393ac634ebdaf81fa58432e0422)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:33.957372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: WARNING py.warnings [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:13:33.957372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: warnings.warn(msg) Sep 10 14:13:33.957372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: Sep 10 14:13:33.958949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:34.087755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518d6393ac634ebdaf81fa58432e0422', 'request_id': 'req-8882b61b-5e9e-434a-9c4a-914cd4be0e98'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:34.090876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:34.100439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518d6393ac634ebdaf81fa58432e0422', 'request_id': 'req-8882b61b-5e9e-434a-9c4a-914cd4be0e98'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:34.113082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518d6393ac634ebdaf81fa58432e0422', 'request_id': 'req-8882b61b-5e9e-434a-9c4a-914cd4be0e98'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:34.162787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] The token cache is being invalidate because project 518d6393ac634ebdaf81fa58432e0422 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:13:34.163325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:34.164095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8882b61b-5e9e-434a-9c4a-914cd4be0e98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:34.166058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 825/1648] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:13:33 2022] DELETE /identity/v3/projects/518d6393ac634ebdaf81fa58432e0422 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:34.172208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb2700-95c6-4e79-93d1-31d2d5830f03 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:34.172757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb2700-95c6-4e79-93d1-31d2d5830f03 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:34.172976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb2700-95c6-4e79-93d1-31d2d5830f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:34.209428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9ceb2700-95c6-4e79-93d1-31d2d5830f03 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:34.210815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9ceb2700-95c6-4e79-93d1-31d2d5830f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:34.332582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:34.358680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 824/1649] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:13:34.367074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af98258-674b-4ae7-a2c5-442df9191257 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:34.367701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af98258-674b-4ae7-a2c5-442df9191257 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:34.368246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af98258-674b-4ae7-a2c5-442df9191257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:34.416540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-4af98258-674b-4ae7-a2c5-442df9191257 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:34.418222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4af98258-674b-4ae7-a2c5-442df9191257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:34.454643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 826/1650] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:34.458340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb2700-95c6-4e79-93d1-31d2d5830f03 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:34.474459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb2700-95c6-4e79-93d1-31d2d5830f03 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:34.504519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d50abd-954a-488f-a1d2-2ba5fac287ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:34.508813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d50abd-954a-488f-a1d2-2ba5fac287ab admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:34.509335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d50abd-954a-488f-a1d2-2ba5fac287ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:34.509769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d50abd-954a-488f-a1d2-2ba5fac287ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:34.512074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d50abd-954a-488f-a1d2-2ba5fac287ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:34.512570 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d50abd-954a-488f-a1d2-2ba5fac287ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:34.518913 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d50abd-954a-488f-a1d2-2ba5fac287ab admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:34.547717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 825/1651] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:34.552247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af98258-674b-4ae7-a2c5-442df9191257 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:34.567359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af98258-674b-4ae7-a2c5-442df9191257 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:34.600227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edd52e-c02a-41fa-b597-b0876d3ab7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:34.600861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edd52e-c02a-41fa-b597-b0876d3ab7e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:34.601142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edd52e-c02a-41fa-b597-b0876d3ab7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:34.601413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edd52e-c02a-41fa-b597-b0876d3ab7e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:34.602059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edd52e-c02a-41fa-b597-b0876d3ab7e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:34.604071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edd52e-c02a-41fa-b597-b0876d3ab7e7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:34.671973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 827/1652] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:34 2022] POST /identity/v3/projects => generated 363 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:13:34.676369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d50abd-954a-488f-a1d2-2ba5fac287ab admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:34.690381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d50abd-954a-488f-a1d2-2ba5fac287ab admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:34.837476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fea85-ba25-4072-ac85-95d29036d003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:34.838445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fea85-ba25-4072-ac85-95d29036d003 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:34.838829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fea85-ba25-4072-ac85-95d29036d003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:34.840646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766fea85-ba25-4072-ac85-95d29036d003 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:34.840646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fea85-ba25-4072-ac85-95d29036d003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:34.843590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766fea85-ba25-4072-ac85-95d29036d003 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:34.899708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 826/1653] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:34 2022] POST /identity/v3/users => generated 282 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:34.903692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edd52e-c02a-41fa-b597-b0876d3ab7e7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:34.923113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edd52e-c02a-41fa-b597-b0876d3ab7e7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:34.941368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1be64d-14e6-4dbd-91be-5ac7b43052c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:34.942012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1be64d-14e6-4dbd-91be-5ac7b43052c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:34.942313 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1be64d-14e6-4dbd-91be-5ac7b43052c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:34.942552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1be64d-14e6-4dbd-91be-5ac7b43052c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:34.943206 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1be64d-14e6-4dbd-91be-5ac7b43052c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:34.944897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1be64d-14e6-4dbd-91be-5ac7b43052c9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:34.979060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 828/1654] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:34.984013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fea85-ba25-4072-ac85-95d29036d003 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:34.999175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766fea85-ba25-4072-ac85-95d29036d003 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:35.006965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854fd533-7750-4985-9d47-06b2916c7645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:35.007675 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fd533-7750-4985-9d47-06b2916c7645 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:35.007986 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fd533-7750-4985-9d47-06b2916c7645 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:35.008909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854fd533-7750-4985-9d47-06b2916c7645 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:35.009621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fd533-7750-4985-9d47-06b2916c7645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:35.011244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854fd533-7750-4985-9d47-06b2916c7645 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:35.023529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 827/1655] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:35.027948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1be64d-14e6-4dbd-91be-5ac7b43052c9 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:35.042546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1be64d-14e6-4dbd-91be-5ac7b43052c9 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:35.054242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88709f1-36d6-410c-8cf3-2c578f793714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:35.055750 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88709f1-36d6-410c-8cf3-2c578f793714 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:35.056880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88709f1-36d6-410c-8cf3-2c578f793714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:35.057405 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88709f1-36d6-410c-8cf3-2c578f793714 admin admin] PATH_INFO: `/v3/projects/9caafcd9703e45a9803b054045e11073/users/02a0a3f1302e43aeb904c3581f639027/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:35.090982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88709f1-36d6-410c-8cf3-2c578f793714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9caafcd9703e45a9803b054045e11073, user_id=02a0a3f1302e43aeb904c3581f639027, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:35.095738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88709f1-36d6-410c-8cf3-2c578f793714 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:35.133983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 829/1656] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:35 2022] PUT /identity/v3/projects/9caafcd9703e45a9803b054045e11073/users/02a0a3f1302e43aeb904c3581f639027/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:35.149119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86cea0-971d-4c0c-9ee1-7a9b2521c592 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:35.149782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86cea0-971d-4c0c-9ee1-7a9b2521c592 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:35.150261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86cea0-971d-4c0c-9ee1-7a9b2521c592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:35.224459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5e86cea0-971d-4c0c-9ee1-7a9b2521c592 None None] MFA Rules not processed for user `02a0a3f1302e43aeb904c3581f639027`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:35.228255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5e86cea0-971d-4c0c-9ee1-7a9b2521c592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:35.410711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 828/1657] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:35.415364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88709f1-36d6-410c-8cf3-2c578f793714 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:35.532931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88709f1-36d6-410c-8cf3-2c578f793714 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:35.546980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524c353-1f37-41a7-9f57-63d34ebe4908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:35.547884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524c353-1f37-41a7-9f57-63d34ebe4908 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:35.548327 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524c353-1f37-41a7-9f57-63d34ebe4908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:35.548929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524c353-1f37-41a7-9f57-63d34ebe4908 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:35.550105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524c353-1f37-41a7-9f57-63d34ebe4908 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:35.553278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5524c353-1f37-41a7-9f57-63d34ebe4908 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:35.618666 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 830/1658] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:35 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:35.623203 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86cea0-971d-4c0c-9ee1-7a9b2521c592 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:35.638005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e86cea0-971d-4c0c-9ee1-7a9b2521c592 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:35.748738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:35.760244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3ad95-95a5-4001-9e6a-3f8a70c1747b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:35.761388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3ad95-95a5-4001-9e6a-3f8a70c1747b admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:35.761518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3ad95-95a5-4001-9e6a-3f8a70c1747b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:35.761912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3ad95-95a5-4001-9e6a-3f8a70c1747b admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:35.762703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3ad95-95a5-4001-9e6a-3f8a70c1747b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:35.767358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad3ad95-95a5-4001-9e6a-3f8a70c1747b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:35.805545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 829/1659] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:35 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:35.810924 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524c353-1f37-41a7-9f57-63d34ebe4908 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:35.863057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524c353-1f37-41a7-9f57-63d34ebe4908 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:35.882038 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7728e1-6e7c-41bf-9036-80b88c1e9dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:35.883355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7728e1-6e7c-41bf-9036-80b88c1e9dbd admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:35.884567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7728e1-6e7c-41bf-9036-80b88c1e9dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:35.885758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7728e1-6e7c-41bf-9036-80b88c1e9dbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:35.886502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7728e1-6e7c-41bf-9036-80b88c1e9dbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:35.890473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e7728e1-6e7c-41bf-9036-80b88c1e9dbd admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:35.936172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 831/1660] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:35 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:35.941079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3ad95-95a5-4001-9e6a-3f8a70c1747b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:35.960266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad3ad95-95a5-4001-9e6a-3f8a70c1747b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:35.973192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb941b0-93bb-4972-82eb-88ccf1208d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:35.973993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb941b0-93bb-4972-82eb-88ccf1208d83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:35.975522 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb941b0-93bb-4972-82eb-88ccf1208d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:35.976006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb941b0-93bb-4972-82eb-88ccf1208d83 admin admin] PATH_INFO: `/v3/projects/ceb77638847c4d1195c3dc9567c223b4/users/a62fe42b09a1402f83c78171049ade87/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:36.023924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb941b0-93bb-4972-82eb-88ccf1208d83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb77638847c4d1195c3dc9567c223b4, user_id=a62fe42b09a1402f83c78171049ade87, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:36.026774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb941b0-93bb-4972-82eb-88ccf1208d83 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:36.048690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 830/1661] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:35 2022] PUT /identity/v3/projects/ceb77638847c4d1195c3dc9567c223b4/users/a62fe42b09a1402f83c78171049ade87/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:36.052646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7728e1-6e7c-41bf-9036-80b88c1e9dbd admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:36.075504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7728e1-6e7c-41bf-9036-80b88c1e9dbd admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:36.164872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:36.196990 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e2917-d29d-4a40-a7e9-8dcbb30a6058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:36.197820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e2917-d29d-4a40-a7e9-8dcbb30a6058 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:36.198206 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e2917-d29d-4a40-a7e9-8dcbb30a6058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:36.199016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e2917-d29d-4a40-a7e9-8dcbb30a6058 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:36.200017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e2917-d29d-4a40-a7e9-8dcbb30a6058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:36.202219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e2917-d29d-4a40-a7e9-8dcbb30a6058 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:36.219288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 832/1662] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:36.223269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb941b0-93bb-4972-82eb-88ccf1208d83 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:36.239499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb941b0-93bb-4972-82eb-88ccf1208d83 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:36.246656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df6ab0f-949d-412c-ae36-fee189ae1be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:36.248971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6ab0f-949d-412c-ae36-fee189ae1be6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:36.249611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6ab0f-949d-412c-ae36-fee189ae1be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:36.250406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df6ab0f-949d-412c-ae36-fee189ae1be6 admin admin] PATH_INFO: `/v3/projects/ceb77638847c4d1195c3dc9567c223b4/users/a62fe42b09a1402f83c78171049ade87/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:36.253675 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6ab0f-949d-412c-ae36-fee189ae1be6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceb77638847c4d1195c3dc9567c223b4, user_id=a62fe42b09a1402f83c78171049ade87, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:36.256879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df6ab0f-949d-412c-ae36-fee189ae1be6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:36.290702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 831/1663] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:36 2022] PUT /identity/v3/projects/ceb77638847c4d1195c3dc9567c223b4/users/a62fe42b09a1402f83c78171049ade87/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:36.298012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f849f75-c0f4-4ec1-9f33-50caca6898a7 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:36.298012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f849f75-c0f4-4ec1-9f33-50caca6898a7 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:36.298240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f849f75-c0f4-4ec1-9f33-50caca6898a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:36.338369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2f849f75-c0f4-4ec1-9f33-50caca6898a7 None None] MFA Rules not processed for user `a62fe42b09a1402f83c78171049ade87`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:36.346436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2f849f75-c0f4-4ec1-9f33-50caca6898a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:36.474404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127388-7fc3-4a17-9552-ff28a7357ca7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:36.475826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127388-7fc3-4a17-9552-ff28a7357ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:36.476363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c127388-7fc3-4a17-9552-ff28a7357ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:36.570459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 833/1664] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:36 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:36.594450 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f849f75-c0f4-4ec1-9f33-50caca6898a7 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:36.598627 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2f849f75-c0f4-4ec1-9f33-50caca6898a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:36.605867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6c127388-7fc3-4a17-9552-ff28a7357ca7 None None] MFA Rules not processed for user `f7d4fc9ff073439a94b62638d33381e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:36.609645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6c127388-7fc3-4a17-9552-ff28a7357ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:36.784965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f849f75-c0f4-4ec1-9f33-50caca6898a7 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:36.795499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 832/1665] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:36 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:36.795785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dffe22-8815-4112-8fc6-3280ebfe4a93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:36.795785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dffe22-8815-4112-8fc6-3280ebfe4a93 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:36.795785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dffe22-8815-4112-8fc6-3280ebfe4a93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:36.795785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dffe22-8815-4112-8fc6-3280ebfe4a93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:36.838762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dffe22-8815-4112-8fc6-3280ebfe4a93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:36.843042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dffe22-8815-4112-8fc6-3280ebfe4a93 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:36.866217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 834/1666] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:37.912675 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c127388-7fc3-4a17-9552-ff28a7357ca7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:38.087246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c127388-7fc3-4a17-9552-ff28a7357ca7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:38.112587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:38.117143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:38.117746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:38.118523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] PATH_INFO: `/v3/users/02a0a3f1302e43aeb904c3581f639027` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:38.124696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] RBAC: Authorizing `identity:delete_user(user_id=02a0a3f1302e43aeb904c3581f639027)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:38.126032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: WARNING py.warnings [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 10 14:13:38.126032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: warnings.warn(msg) Sep 10 14:13:38.126032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: Sep 10 14:13:38.128014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:38.236455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc65f8-b062-4153-aeaf-7ca0fbf74baf None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:38.236910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc65f8-b062-4153-aeaf-7ca0fbf74baf None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:38.237089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc65f8-b062-4153-aeaf-7ca0fbf74baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:38.257735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:38.300219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02a0a3f1302e43aeb904c3581f639027'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:38.311009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-0abc65f8-b062-4153-aeaf-7ca0fbf74baf None None] MFA Rules not processed for user `db2f1171c5394a1c8879cd09c1e685b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:38.311597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0abc65f8-b062-4153-aeaf-7ca0fbf74baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:38.314567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02a0a3f1302e43aeb904c3581f639027'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:38.315264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02a0a3f1302e43aeb904c3581f639027'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:38.315969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02a0a3f1302e43aeb904c3581f639027'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:38.339553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fca631da-0d39-4e10-a884-16e923f2670d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02a0a3f1302e43aeb904c3581f639027'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:38.403175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 833/1667] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:13:37 2022] DELETE /identity/v3/users/02a0a3f1302e43aeb904c3581f639027 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:38.408461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbf56c-c371-4076-b36c-9ce7ee63a626 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:38.408767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbf56c-c371-4076-b36c-9ce7ee63a626 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:38.408924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babbf56c-c371-4076-b36c-9ce7ee63a626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:38.452836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-babbf56c-c371-4076-b36c-9ce7ee63a626 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:38.455255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-babbf56c-c371-4076-b36c-9ce7ee63a626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:38.467497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 835/1668] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:38.483609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc65f8-b062-4153-aeaf-7ca0fbf74baf None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:38.492465 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0abc65f8-b062-4153-aeaf-7ca0fbf74baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:38.500521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:38.577315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 834/1669] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:13:38.652426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abc65f8-b062-4153-aeaf-7ca0fbf74baf None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:38.669045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbf56c-c371-4076-b36c-9ce7ee63a626 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:38.678368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-babbf56c-c371-4076-b36c-9ce7ee63a626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:38.864056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babbf56c-c371-4076-b36c-9ce7ee63a626 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:38.871183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7e6862-fe9a-48f0-bc19-bdf86cd327ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:38.871748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f75442-ed05-4c9a-9840-2c9cc7606027 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:38.872287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e6862-fe9a-48f0-bc19-bdf86cd327ed service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:38.872287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e6862-fe9a-48f0-bc19-bdf86cd327ed service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:38.872846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7e6862-fe9a-48f0-bc19-bdf86cd327ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:38.874754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f75442-ed05-4c9a-9840-2c9cc7606027 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:38.875257 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f75442-ed05-4c9a-9840-2c9cc7606027 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:38.875941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f75442-ed05-4c9a-9840-2c9cc7606027 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:38.901510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e6862-fe9a-48f0-bc19-bdf86cd327ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:38.906164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7e6862-fe9a-48f0-bc19-bdf86cd327ed service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:38.934579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 835/1670] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:13:39.002863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f75442-ed05-4c9a-9840-2c9cc7606027 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:39.004707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f75442-ed05-4c9a-9840-2c9cc7606027 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:39.044154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 836/1671] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:13:39.815240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c24e7-0d40-456a-bf7a-34017d2b38c9 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:39.815617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c24e7-0d40-456a-bf7a-34017d2b38c9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:39.815903 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247c24e7-0d40-456a-bf7a-34017d2b38c9 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:39.817309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 836/1672] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:13:39.859501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4516c26-f468-422e-8310-302d2ce2a47e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:39.860055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4516c26-f468-422e-8310-302d2ce2a47e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:39.860055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4516c26-f468-422e-8310-302d2ce2a47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:39.907518 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-b4516c26-f468-422e-8310-302d2ce2a47e None None] MFA Rules not processed for user `2a6a70a10cfa49348cc8b2b42870909d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:39.909031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b4516c26-f468-422e-8310-302d2ce2a47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:40.015361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:40.085117 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 837/1673] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:13:39 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:13:40.160739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c24e7-0d40-456a-bf7a-34017d2b38c9 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:40.162741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-247c24e7-0d40-456a-bf7a-34017d2b38c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:40.196430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247c24e7-0d40-456a-bf7a-34017d2b38c9 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:40.210632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3eb01-f654-478d-8813-6b95439535ec service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:40.212451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3eb01-f654-478d-8813-6b95439535ec service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:40.212965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3eb01-f654-478d-8813-6b95439535ec service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:40.213454 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf3eb01-f654-478d-8813-6b95439535ec service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:40.233126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3eb01-f654-478d-8813-6b95439535ec service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:40.238856 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf3eb01-f654-478d-8813-6b95439535ec service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:40.263970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 837/1674] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:41.269759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4516c26-f468-422e-8310-302d2ce2a47e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:41.272260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b4516c26-f468-422e-8310-302d2ce2a47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:41.299007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4516c26-f468-422e-8310-302d2ce2a47e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:41.315474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:41.316614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:41.317160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:41.317642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] PATH_INFO: `/v3/projects/9caafcd9703e45a9803b054045e11073` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:41.319860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9caafcd9703e45a9803b054045e11073)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:41.324523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:41.471679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9caafcd9703e45a9803b054045e11073', 'request_id': 'req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:41.490070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9caafcd9703e45a9803b054045e11073', 'request_id': 'req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:41.498682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9caafcd9703e45a9803b054045e11073', 'request_id': 'req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:41.527075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] The token cache is being invalidate because project 9caafcd9703e45a9803b054045e11073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:13:41.527715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:41.528724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f2aea75b-a856-4d95-8388-f8e8b01f3e4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:41.530796 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 838/1675] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:13:41 2022] DELETE /identity/v3/projects/9caafcd9703e45a9803b054045e11073 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:13:41.534599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3eb01-f654-478d-8813-6b95439535ec service placement] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:41.552725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf3eb01-f654-478d-8813-6b95439535ec service placement] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:41.661984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:41.686119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:41.686834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:41.687163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:41.687618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] PATH_INFO: `/v3/users/a62fe42b09a1402f83c78171049ade87` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:41.688916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] RBAC: Authorizing `identity:delete_user(user_id=a62fe42b09a1402f83c78171049ade87)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:41.691246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:41.803425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a62fe42b09a1402f83c78171049ade87'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:41.827165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a62fe42b09a1402f83c78171049ade87'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:41.828592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a62fe42b09a1402f83c78171049ade87'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:41.829680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a62fe42b09a1402f83c78171049ade87'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:41.862694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3c7e1ee7-d43d-4b50-bd05-0c532e9d1fad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a62fe42b09a1402f83c78171049ade87'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:41.918524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 838/1676] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:13:41 2022] DELETE /identity/v3/users/a62fe42b09a1402f83c78171049ade87 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:44.041151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378f88-4442-4808-8466-df0fb1b65dec None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:44.042215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378f88-4442-4808-8466-df0fb1b65dec None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:44.042804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a378f88-4442-4808-8466-df0fb1b65dec None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:44.045861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 839/1677] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:13:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:13:44.097867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69403327-b423-45fe-9baf-e9804d128f08 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:44.098315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69403327-b423-45fe-9baf-e9804d128f08 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:44.098503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69403327-b423-45fe-9baf-e9804d128f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:44.139754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-69403327-b423-45fe-9baf-e9804d128f08 None None] MFA Rules not processed for user `2a6a70a10cfa49348cc8b2b42870909d`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:44.144702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-69403327-b423-45fe-9baf-e9804d128f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:44.320127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 839/1678] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:13:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:44.392058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a378f88-4442-4808-8466-df0fb1b65dec None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:44.394331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0a378f88-4442-4808-8466-df0fb1b65dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:44.448751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a378f88-4442-4808-8466-df0fb1b65dec None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:44.451902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69403327-b423-45fe-9baf-e9804d128f08 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:44.453327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-69403327-b423-45fe-9baf-e9804d128f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:44.479116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3db4e-59bb-480d-90b3-f5f17bdb13a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:44.483415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3db4e-59bb-480d-90b3-f5f17bdb13a0 service placement] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:44.483729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3db4e-59bb-480d-90b3-f5f17bdb13a0 service placement] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:44.486516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb3db4e-59bb-480d-90b3-f5f17bdb13a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:44.537479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3db4e-59bb-480d-90b3-f5f17bdb13a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:44.543542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb3db4e-59bb-480d-90b3-f5f17bdb13a0 service placement] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:44.588778 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 840/1679] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:44.674627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:44.685285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69403327-b423-45fe-9baf-e9804d128f08 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:44.707312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:44.708011 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:44.708293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:44.708569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] PATH_INFO: `/v3/projects/ceb77638847c4d1195c3dc9567c223b4` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:44.711846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceb77638847c4d1195c3dc9567c223b4)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:44.714093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:44.746590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3db4e-59bb-480d-90b3-f5f17bdb13a0 service placement] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:44.748378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4fb3db4e-59bb-480d-90b3-f5f17bdb13a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:44.785487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb3db4e-59bb-480d-90b3-f5f17bdb13a0 service placement] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:44.806038 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:44.806749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:44.807030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:44.812276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] PATH_INFO: `/v3/users/8afdfd824d604b2f8f0f56d3f2c158bf` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:44.815083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8afdfd824d604b2f8f0f56d3f2c158bf)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:44.821412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:44.873602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceb77638847c4d1195c3dc9567c223b4', 'request_id': 'req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:44.883641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceb77638847c4d1195c3dc9567c223b4', 'request_id': 'req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:44.906772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceb77638847c4d1195c3dc9567c223b4', 'request_id': 'req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:44.943659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] The token cache is being invalidate because project ceb77638847c4d1195c3dc9567c223b4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:13:44.944914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:44.945888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ce6f3f73-2ec1-4f39-9b62-20d1f81d798b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:44.948138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8afdfd824d604b2f8f0f56d3f2c158bf'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:44.949563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8afdfd824d604b2f8f0f56d3f2c158bf'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:44.962453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 840/1680] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:13:44 2022] DELETE /identity/v3/projects/ceb77638847c4d1195c3dc9567c223b4 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:44.963906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18da092-7914-459d-812e-99ca36f9e733 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:44.964199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18da092-7914-459d-812e-99ca36f9e733 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:44.964485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18da092-7914-459d-812e-99ca36f9e733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:44.978452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8afdfd824d604b2f8f0f56d3f2c158bf'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:45.003728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8afdfd824d604b2f8f0f56d3f2c158bf'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:45.005325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c464d9f4-30b8-4a18-8c71-f63ce13fa55b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8afdfd824d604b2f8f0f56d3f2c158bf'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:45.007951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f18da092-7914-459d-812e-99ca36f9e733 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:45.009210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f18da092-7914-459d-812e-99ca36f9e733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:45.048872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 841/1681] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:13:44 2022] DELETE /identity/v3/users/8afdfd824d604b2f8f0f56d3f2c158bf => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:13:45.171903 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 841/1682] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:13:45.179367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc5ee1-c81b-42b7-8fce-3f4f90e9bf4e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:45.180870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc5ee1-c81b-42b7-8fce-3f4f90e9bf4e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:45.180870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc5ee1-c81b-42b7-8fce-3f4f90e9bf4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:45.219697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-34dc5ee1-c81b-42b7-8fce-3f4f90e9bf4e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:45.221396 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-34dc5ee1-c81b-42b7-8fce-3f4f90e9bf4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:45.255918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:45.391158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 842/1683] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:45.395681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18da092-7914-459d-812e-99ca36f9e733 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:45.408366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18da092-7914-459d-812e-99ca36f9e733 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:45.421046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a87ac-e54f-4178-a0be-c30732683df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:45.421799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a87ac-e54f-4178-a0be-c30732683df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:45.422166 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a87ac-e54f-4178-a0be-c30732683df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:45.422902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a87ac-e54f-4178-a0be-c30732683df3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:45.424165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a87ac-e54f-4178-a0be-c30732683df3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:45.424165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a87ac-e54f-4178-a0be-c30732683df3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:45.430291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5a87ac-e54f-4178-a0be-c30732683df3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:45.465148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 842/1684] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:45.471104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc5ee1-c81b-42b7-8fce-3f4f90e9bf4e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:45.481610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc5ee1-c81b-42b7-8fce-3f4f90e9bf4e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:45.498028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab64d5-b9bd-49f7-9dc1-f3fa00cb6657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:45.498703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab64d5-b9bd-49f7-9dc1-f3fa00cb6657 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:45.498985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab64d5-b9bd-49f7-9dc1-f3fa00cb6657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:45.499262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab64d5-b9bd-49f7-9dc1-f3fa00cb6657 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:45.499932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab64d5-b9bd-49f7-9dc1-f3fa00cb6657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:45.502324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab64d5-b9bd-49f7-9dc1-f3fa00cb6657 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:45.562653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 843/1685] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:45 2022] POST /identity/v3/projects => generated 363 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:45.566612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a87ac-e54f-4178-a0be-c30732683df3 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:45.586083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a87ac-e54f-4178-a0be-c30732683df3 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:45.728028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea56710-44e0-487d-9217-43509b74f2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:45.728871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea56710-44e0-487d-9217-43509b74f2ce admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:45.729161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea56710-44e0-487d-9217-43509b74f2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:45.729447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea56710-44e0-487d-9217-43509b74f2ce admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:45.730943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea56710-44e0-487d-9217-43509b74f2ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:45.735189 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea56710-44e0-487d-9217-43509b74f2ce admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:45.785883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 843/1686] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:45 2022] POST /identity/v3/users => generated 282 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:45.787100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab64d5-b9bd-49f7-9dc1-f3fa00cb6657 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:45.817769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab64d5-b9bd-49f7-9dc1-f3fa00cb6657 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:45.833743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6cf79-3976-498d-81bc-62abfbcb652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:45.834631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6cf79-3976-498d-81bc-62abfbcb652e admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:45.837154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6cf79-3976-498d-81bc-62abfbcb652e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:45.837441 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c6cf79-3976-498d-81bc-62abfbcb652e admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:45.843167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6cf79-3976-498d-81bc-62abfbcb652e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:45.855331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c6cf79-3976-498d-81bc-62abfbcb652e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:45.902348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 844/1687] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:45.903608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea56710-44e0-487d-9217-43509b74f2ce admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:45.911859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:45.917775 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea56710-44e0-487d-9217-43509b74f2ce admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:45.927970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb87b07-be2d-46e7-8b11-6ee5471bfbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:45.929258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb87b07-be2d-46e7-8b11-6ee5471bfbfb admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:45.931129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb87b07-be2d-46e7-8b11-6ee5471bfbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:45.932317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb87b07-be2d-46e7-8b11-6ee5471bfbfb admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:45.937024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb87b07-be2d-46e7-8b11-6ee5471bfbfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:45.940926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb87b07-be2d-46e7-8b11-6ee5471bfbfb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:45.979875 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6cf79-3976-498d-81bc-62abfbcb652e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:45.982645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 844/1688] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:45.992589 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c6cf79-3976-498d-81bc-62abfbcb652e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:46.012523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41a04d-7c96-4bb6-8d6e-a35df6b7029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:46.013236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41a04d-7c96-4bb6-8d6e-a35df6b7029e admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:46.013517 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41a04d-7c96-4bb6-8d6e-a35df6b7029e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:46.013797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a41a04d-7c96-4bb6-8d6e-a35df6b7029e admin admin] PATH_INFO: `/v3/projects/fed001640cb44b989c8e4a486bc795ec/users/3c90d7bd370b4adb9cd5921846544431/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:46.055403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41a04d-7c96-4bb6-8d6e-a35df6b7029e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fed001640cb44b989c8e4a486bc795ec, user_id=3c90d7bd370b4adb9cd5921846544431, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:46.058142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a41a04d-7c96-4bb6-8d6e-a35df6b7029e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:46.138915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 845/1689] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:45 2022] PUT /identity/v3/projects/fed001640cb44b989c8e4a486bc795ec/users/3c90d7bd370b4adb9cd5921846544431/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:13:46.140826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c515-a175-4e43-ade1-09a10a89af27 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:46.143343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c515-a175-4e43-ade1-09a10a89af27 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:46.143724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c758c515-a175-4e43-ade1-09a10a89af27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:46.241277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c758c515-a175-4e43-ade1-09a10a89af27 None None] MFA Rules not processed for user `3c90d7bd370b4adb9cd5921846544431`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:46.244509 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c758c515-a175-4e43-ade1-09a10a89af27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:46.363677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 845/1690] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:46.364943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41a04d-7c96-4bb6-8d6e-a35df6b7029e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:46.458020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13ed1a-f5b1-4641-8848-8fe34609b63f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:46.458463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13ed1a-f5b1-4641-8848-8fe34609b63f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:46.458860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13ed1a-f5b1-4641-8848-8fe34609b63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:46.492830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5b13ed1a-f5b1-4641-8848-8fe34609b63f None None] MFA Rules not processed for user `db2f1171c5394a1c8879cd09c1e685b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:46.497982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5b13ed1a-f5b1-4641-8848-8fe34609b63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:46.517250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a41a04d-7c96-4bb6-8d6e-a35df6b7029e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:46.531834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de155db9-0ba0-46e7-a0ac-ad86a53f5825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:46.533826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de155db9-0ba0-46e7-a0ac-ad86a53f5825 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:46.534474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de155db9-0ba0-46e7-a0ac-ad86a53f5825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:46.535179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de155db9-0ba0-46e7-a0ac-ad86a53f5825 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:46.536227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de155db9-0ba0-46e7-a0ac-ad86a53f5825 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:46.538596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de155db9-0ba0-46e7-a0ac-ad86a53f5825 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:46.604024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 846/1691] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:46 2022] POST /identity/v3/projects => generated 363 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:46.605006 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de155db9-0ba0-46e7-a0ac-ad86a53f5825 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:46.628667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 846/1692] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:46.679376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:46.704360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de155db9-0ba0-46e7-a0ac-ad86a53f5825 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:46.711678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166413b6-93d4-4dc6-8095-721d92a0acd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:46.712109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166413b6-93d4-4dc6-8095-721d92a0acd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:46.712487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166413b6-93d4-4dc6-8095-721d92a0acd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:46.712856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166413b6-93d4-4dc6-8095-721d92a0acd2 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:46.713609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166413b6-93d4-4dc6-8095-721d92a0acd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:46.715983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166413b6-93d4-4dc6-8095-721d92a0acd2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:46.757986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 847/1693] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:46 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:46.761693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13ed1a-f5b1-4641-8848-8fe34609b63f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:46.785789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13ed1a-f5b1-4641-8848-8fe34609b63f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:46.801165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866da4a5-0493-4232-871b-80f883d7192b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:46.801810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866da4a5-0493-4232-871b-80f883d7192b admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:46.802193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866da4a5-0493-4232-871b-80f883d7192b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:46.802468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866da4a5-0493-4232-871b-80f883d7192b admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:46.803229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866da4a5-0493-4232-871b-80f883d7192b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:46.805055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866da4a5-0493-4232-871b-80f883d7192b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:46.835656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 847/1694] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:46.837441 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166413b6-93d4-4dc6-8095-721d92a0acd2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:46.871369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166413b6-93d4-4dc6-8095-721d92a0acd2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:46.896474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385305e-b87a-44da-a62e-fa069a5f3f0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:46.897168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385305e-b87a-44da-a62e-fa069a5f3f0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:46.897450 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385305e-b87a-44da-a62e-fa069a5f3f0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:46.897755 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385305e-b87a-44da-a62e-fa069a5f3f0b admin admin] PATH_INFO: `/v3/projects/9a6373074cbe471ab79e026e6ac40b84/users/5cedd0cc77fd4eb7831cd1173e65f77e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:46.966225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc476-2b76-4d97-bb72-004b8cb4f02e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:46.967676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc476-2b76-4d97-bb72-004b8cb4f02e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:46.968281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45cc476-2b76-4d97-bb72-004b8cb4f02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:46.973697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385305e-b87a-44da-a62e-fa069a5f3f0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6373074cbe471ab79e026e6ac40b84, user_id=5cedd0cc77fd4eb7831cd1173e65f77e, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:46.991503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a385305e-b87a-44da-a62e-fa069a5f3f0b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:47.035725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 848/1695] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:46 2022] PUT /identity/v3/projects/9a6373074cbe471ab79e026e6ac40b84/users/5cedd0cc77fd4eb7831cd1173e65f77e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:47.035725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385305e-b87a-44da-a62e-fa069a5f3f0b admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:47.037988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a385305e-b87a-44da-a62e-fa069a5f3f0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:47.061969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a45cc476-2b76-4d97-bb72-004b8cb4f02e None None] MFA Rules not processed for user `8ea8ba61dfa54cbc9a45543d20f31e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:47.063213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a45cc476-2b76-4d97-bb72-004b8cb4f02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:47.241787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385305e-b87a-44da-a62e-fa069a5f3f0b admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:47.254552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:47.255285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:47.255597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:47.255871 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] PATH_INFO: `/v3/projects/1f7040e4e07a41318f060710509c6e54` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:47.268211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f7040e4e07a41318f060710509c6e54)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:47.268422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 848/1696] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:47.268422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc476-2b76-4d97-bb72-004b8cb4f02e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:47.268916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:47.299672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45cc476-2b76-4d97-bb72-004b8cb4f02e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:47.318271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8e71e-047b-443d-a83f-5d7b17520f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:47.322056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8e71e-047b-443d-a83f-5d7b17520f56 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:47.322348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8e71e-047b-443d-a83f-5d7b17520f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:47.322581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e8e71e-047b-443d-a83f-5d7b17520f56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:47.328307 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8e71e-047b-443d-a83f-5d7b17520f56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:47.336012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e8e71e-047b-443d-a83f-5d7b17520f56 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:47.337805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:47.375547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 849/1697] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:47.382177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8e71e-047b-443d-a83f-5d7b17520f56 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:47.386266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-56e8e71e-047b-443d-a83f-5d7b17520f56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:47.409002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f7040e4e07a41318f060710509c6e54', 'request_id': 'req-4332a74a-5bb7-4330-a145-c2bb460a03af'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:47.430903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f7040e4e07a41318f060710509c6e54', 'request_id': 'req-4332a74a-5bb7-4330-a145-c2bb460a03af'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:47.446844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f7040e4e07a41318f060710509c6e54', 'request_id': 'req-4332a74a-5bb7-4330-a145-c2bb460a03af'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:47.473618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] The token cache is being invalidate because project 1f7040e4e07a41318f060710509c6e54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:13:47.473955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:47.474727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:47.478394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 849/1698] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:13:47 2022] DELETE /identity/v3/projects/1f7040e4e07a41318f060710509c6e54 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:47.484576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:47.489672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:47.659271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e8e71e-047b-443d-a83f-5d7b17520f56 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:47.661677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:47.710369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4332a74a-5bb7-4330-a145-c2bb460a03af admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:47.717729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dadd54-b28f-41be-9413-c5fc6200225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:47.718803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dadd54-b28f-41be-9413-c5fc6200225c admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:47.719082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dadd54-b28f-41be-9413-c5fc6200225c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:47.719356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dadd54-b28f-41be-9413-c5fc6200225c admin admin] PATH_INFO: `/v3/projects/9a6373074cbe471ab79e026e6ac40b84/users/5cedd0cc77fd4eb7831cd1173e65f77e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:47.722865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dadd54-b28f-41be-9413-c5fc6200225c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a6373074cbe471ab79e026e6ac40b84, user_id=5cedd0cc77fd4eb7831cd1173e65f77e, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:47.726613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dadd54-b28f-41be-9413-c5fc6200225c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:47.754543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 850/1699] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:47 2022] PUT /identity/v3/projects/9a6373074cbe471ab79e026e6ac40b84/users/5cedd0cc77fd4eb7831cd1173e65f77e/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:47.754543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dadd54-b28f-41be-9413-c5fc6200225c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:47.754543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-97dadd54-b28f-41be-9413-c5fc6200225c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:47.846771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9d9ef4-308e-428a-b37a-90d37ed48401 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:47.848536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d9ef4-308e-428a-b37a-90d37ed48401 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:47.848828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d9ef4-308e-428a-b37a-90d37ed48401 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:47.849547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9d9ef4-308e-428a-b37a-90d37ed48401 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:47.858751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7e9d9ef4-308e-428a-b37a-90d37ed48401 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:47.910651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dadd54-b28f-41be-9413-c5fc6200225c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:47.930066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:47.932043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:47.932332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:47.932658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] PATH_INFO: `/v3/users/f7d4fc9ff073439a94b62638d33381e5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:47.936814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7d4fc9ff073439a94b62638d33381e5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:47.939455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:48.053186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d9ef4-308e-428a-b37a-90d37ed48401 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:48.053961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9d9ef4-308e-428a-b37a-90d37ed48401 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:48.074601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7d4fc9ff073439a94b62638d33381e5'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:48.077154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7d4fc9ff073439a94b62638d33381e5'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:48.080789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 850/1700] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:48.082406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc05607-217c-4ca7-b2c1-fbab0e66b76f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:48.082927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc05607-217c-4ca7-b2c1-fbab0e66b76f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:48.083423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc05607-217c-4ca7-b2c1-fbab0e66b76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:48.088985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7d4fc9ff073439a94b62638d33381e5'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:48.103501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7d4fc9ff073439a94b62638d33381e5'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:48.110466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7d4fc9ff073439a94b62638d33381e5'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:48.132488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5bc05607-217c-4ca7-b2c1-fbab0e66b76f None None] MFA Rules not processed for user `5cedd0cc77fd4eb7831cd1173e65f77e`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:48.137352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5bc05607-217c-4ca7-b2c1-fbab0e66b76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:48.178306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:48.185491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 851/1701] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:13:47 2022] DELETE /identity/v3/users/f7d4fc9ff073439a94b62638d33381e5 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:48.325065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 851/1702] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:48.333823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:48.335623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93307b7-55cd-45b6-8fb1-39000272733c None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:48.335980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93307b7-55cd-45b6-8fb1-39000272733c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:48.336315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93307b7-55cd-45b6-8fb1-39000272733c None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:48.337265 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:48.338090 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 852/1703] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 10 14:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:13:48.360873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113611f9-e845-46a4-9bdf-401fba37f382 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:48.360873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113611f9-e845-46a4-9bdf-401fba37f382 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:48.361220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113611f9-e845-46a4-9bdf-401fba37f382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:48.455013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-113611f9-e845-46a4-9bdf-401fba37f382 None None] MFA Rules not processed for user `12c71caf31354ec193e3e2da231f1e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:48.456372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-113611f9-e845-46a4-9bdf-401fba37f382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:48.481731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89000d8b-498b-484d-a927-4c90f5a40e48 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:48.488308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8f316-0236-4bea-bc86-a59b127cbbac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:48.490127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8f316-0236-4bea-bc86-a59b127cbbac service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:48.491617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8f316-0236-4bea-bc86-a59b127cbbac service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:48.492220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec8f316-0236-4bea-bc86-a59b127cbbac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:48.505121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:48.586642 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 853/1704] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:13:48 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:13:48.600954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec8f316-0236-4bea-bc86-a59b127cbbac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:48.602716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec8f316-0236-4bea-bc86-a59b127cbbac service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:48.622243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 852/1705] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:48 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:48.642136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113611f9-e845-46a4-9bdf-401fba37f382 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:48.643929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-113611f9-e845-46a4-9bdf-401fba37f382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:48.661737 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113611f9-e845-46a4-9bdf-401fba37f382 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:48.672073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbc3c5-e731-4e1f-8ee5-40023a5d9c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:48.672710 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbc3c5-e731-4e1f-8ee5-40023a5d9c9d service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:48.672986 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbc3c5-e731-4e1f-8ee5-40023a5d9c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:48.673266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bbc3c5-e731-4e1f-8ee5-40023a5d9c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:48.696076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbc3c5-e731-4e1f-8ee5-40023a5d9c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:48.697792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bbc3c5-e731-4e1f-8ee5-40023a5d9c9d service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:48.722318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 854/1706] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:48.752761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8f316-0236-4bea-bc86-a59b127cbbac service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:48.762273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec8f316-0236-4bea-bc86-a59b127cbbac service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:48.771762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d071c-6ec1-4343-aec1-07e94ed8df67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:48.772387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d071c-6ec1-4343-aec1-07e94ed8df67 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:48.772665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d071c-6ec1-4343-aec1-07e94ed8df67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:48.772937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d071c-6ec1-4343-aec1-07e94ed8df67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:48.791002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-933d071c-6ec1-4343-aec1-07e94ed8df67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:48.968724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d071c-6ec1-4343-aec1-07e94ed8df67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:48.970911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d071c-6ec1-4343-aec1-07e94ed8df67 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:48.989827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 853/1707] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:49.538463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbc3c5-e731-4e1f-8ee5-40023a5d9c9d service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:49.542613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-43bbc3c5-e731-4e1f-8ee5-40023a5d9c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:49.734126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bbc3c5-e731-4e1f-8ee5-40023a5d9c9d service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:49.744474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:49.745396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:49.745396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:49.745676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] PATH_INFO: `/v3/projects/d4b16a80e9c0448a9e8382c229390678` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:49.750373 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4b16a80e9c0448a9e8382c229390678)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:49.756691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:49.904067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:49.913997 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4b16a80e9c0448a9e8382c229390678', 'request_id': 'req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:49.925094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4b16a80e9c0448a9e8382c229390678', 'request_id': 'req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:49.951376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4b16a80e9c0448a9e8382c229390678', 'request_id': 'req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:49.984187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8892a-9b6b-40c0-a7c7-9804c6626750 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:49.984575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8892a-9b6b-40c0-a7c7-9804c6626750 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:49.984842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8892a-9b6b-40c0-a7c7-9804c6626750 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:49.999788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] The token cache is being invalidate because project d4b16a80e9c0448a9e8382c229390678 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:13:50.004782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:50.009852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc7bd2f7-0fda-4b6d-8cce-ed9de02c2b09 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:50.012581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 855/1708] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:13:49 2022] DELETE /identity/v3/projects/d4b16a80e9c0448a9e8382c229390678 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:50.019852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94382fb9-b1c5-49f9-9db1-1667d1484618 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:50.020087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94382fb9-b1c5-49f9-9db1-1667d1484618 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:50.020340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94382fb9-b1c5-49f9-9db1-1667d1484618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:50.064319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-94382fb9-b1c5-49f9-9db1-1667d1484618 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:50.065663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-94382fb9-b1c5-49f9-9db1-1667d1484618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:50.073430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-06e8892a-9b6b-40c0-a7c7-9804c6626750 None None] MFA Rules not processed for user `810f0fa1f7774b37a09a91964c40b823`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:50.074665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-06e8892a-9b6b-40c0-a7c7-9804c6626750 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:50.250057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 856/1709] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:50.252358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf5200-c2de-47c4-af09-c3e5624895ac None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:50.252650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf5200-c2de-47c4-af09-c3e5624895ac None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:50.252927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf5200-c2de-47c4-af09-c3e5624895ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:50.328983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5edf5200-c2de-47c4-af09-c3e5624895ac None None] MFA Rules not processed for user `5ca58aca3f9d4d8fa9f1c6315db7f54d`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:50.333316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5edf5200-c2de-47c4-af09-c3e5624895ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:50.392577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 854/1710] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:50.399999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604a932-5aed-46b9-ae50-a6071b338203 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:50.400272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604a932-5aed-46b9-ae50-a6071b338203 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:50.400536 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0604a932-5aed-46b9-ae50-a6071b338203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:50.475941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-0604a932-5aed-46b9-ae50-a6071b338203 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:50.477216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0604a932-5aed-46b9-ae50-a6071b338203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:50.520702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 855/1711] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:50.522027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b0788-2944-414c-8d71-46fcc5eaeb95 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:50.522317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b0788-2944-414c-8d71-46fcc5eaeb95 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:50.522577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b0788-2944-414c-8d71-46fcc5eaeb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:50.563306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 857/1712] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:50.565396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf5200-c2de-47c4-af09-c3e5624895ac None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:50.572741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5edf5200-c2de-47c4-af09-c3e5624895ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:50.601672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-152b0788-2944-414c-8d71-46fcc5eaeb95 None None] MFA Rules not processed for user `8f96c9a2a49047b5b8abd3b2c1475a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:50.603295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-152b0788-2944-414c-8d71-46fcc5eaeb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:50.655996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:50.738859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:50.834262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf5200-c2de-47c4-af09-c3e5624895ac None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:50.842573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 856/1713] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:50.843527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b0788-2944-414c-8d71-46fcc5eaeb95 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:50.869114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5aaa0c8-fed8-4007-b5ce-8c5a9db607bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:50.869749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaa0c8-fed8-4007-b5ce-8c5a9db607bb service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:50.870036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaa0c8-fed8-4007-b5ce-8c5a9db607bb service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:50.870303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5aaa0c8-fed8-4007-b5ce-8c5a9db607bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:50.874316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b0788-2944-414c-8d71-46fcc5eaeb95 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:50.894460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f60ebe-a3b9-4dcd-9f3c-d8074e50ae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:50.895090 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f60ebe-a3b9-4dcd-9f3c-d8074e50ae5f admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:50.895362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f60ebe-a3b9-4dcd-9f3c-d8074e50ae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:50.895655 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f60ebe-a3b9-4dcd-9f3c-d8074e50ae5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:50.896323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f60ebe-a3b9-4dcd-9f3c-d8074e50ae5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:50.896623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f60ebe-a3b9-4dcd-9f3c-d8074e50ae5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:50.898262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f60ebe-a3b9-4dcd-9f3c-d8074e50ae5f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:50.921515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 857/1714] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:50.922813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38d6eb-08e8-4b79-8ff3-ead9afbe8b39 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:50.923095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38d6eb-08e8-4b79-8ff3-ead9afbe8b39 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:50.923368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38d6eb-08e8-4b79-8ff3-ead9afbe8b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:50.954511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aaa0c8-fed8-4007-b5ce-8c5a9db607bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:50.957225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5aaa0c8-fed8-4007-b5ce-8c5a9db607bb service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:50.972581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-ec38d6eb-08e8-4b79-8ff3-ead9afbe8b39 None None] MFA Rules not processed for user `5ca58aca3f9d4d8fa9f1c6315db7f54d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:50.973814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ec38d6eb-08e8-4b79-8ff3-ead9afbe8b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:50.984294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 858/1715] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:50.997905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 858/1716] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:51.008649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38d6eb-08e8-4b79-8ff3-ead9afbe8b39 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:51.012416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c458-9dcb-46a3-8032-ecaec89d9f58 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:51.014218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c458-9dcb-46a3-8032-ecaec89d9f58 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:51.014774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77c458-9dcb-46a3-8032-ecaec89d9f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:51.021203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ec38d6eb-08e8-4b79-8ff3-ead9afbe8b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:51.068339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fd77c458-9dcb-46a3-8032-ecaec89d9f58 None None] MFA Rules not processed for user `8f96c9a2a49047b5b8abd3b2c1475a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:51.071265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fd77c458-9dcb-46a3-8032-ecaec89d9f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:51.099838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 859/1717] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:51.101005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c458-9dcb-46a3-8032-ecaec89d9f58 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:51.142144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77c458-9dcb-46a3-8032-ecaec89d9f58 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:51.161917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51ce6f-7cd9-459d-994b-25c16c7a99c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:51.162540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51ce6f-7cd9-459d-994b-25c16c7a99c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:51.162812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51ce6f-7cd9-459d-994b-25c16c7a99c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:51.163077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d51ce6f-7cd9-459d-994b-25c16c7a99c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:51.169658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51ce6f-7cd9-459d-994b-25c16c7a99c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:51.177976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d51ce6f-7cd9-459d-994b-25c16c7a99c1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:51.228935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38d6eb-08e8-4b79-8ff3-ead9afbe8b39 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:51.261114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c842a-b99f-445c-b555-ca38814f56cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:51.265076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c842a-b99f-445c-b555-ca38814f56cf service placement] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:51.265412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c842a-b99f-445c-b555-ca38814f56cf service placement] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:51.265692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986c842a-b99f-445c-b555-ca38814f56cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:51.273662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-986c842a-b99f-445c-b555-ca38814f56cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:51.311114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 860/1718] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:51 2022] POST /identity/v3/projects => generated 369 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:51.316357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51ce6f-7cd9-459d-994b-25c16c7a99c1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:51.319394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c842a-b99f-445c-b555-ca38814f56cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:51.324007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986c842a-b99f-445c-b555-ca38814f56cf service placement] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:51.325875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7d51ce6f-7cd9-459d-994b-25c16c7a99c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:51.368042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 859/1719] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:51.373738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c842a-b99f-445c-b555-ca38814f56cf service placement] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:51.414543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986c842a-b99f-445c-b555-ca38814f56cf service placement] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:51.604019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d51ce6f-7cd9-459d-994b-25c16c7a99c1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:51.624370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa42c37-bd8b-45a6-90b9-f4a038272527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:51.625028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa42c37-bd8b-45a6-90b9-f4a038272527 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:51.626622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa42c37-bd8b-45a6-90b9-f4a038272527 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:51.626918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa42c37-bd8b-45a6-90b9-f4a038272527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:51.636507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112ba66-cbee-4ae2-9572-b275c32991ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:51.637174 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112ba66-cbee-4ae2-9572-b275c32991ec service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:51.639253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112ba66-cbee-4ae2-9572-b275c32991ec service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:51.639708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112ba66-cbee-4ae2-9572-b275c32991ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:51.748945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:51.884141 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112ba66-cbee-4ae2-9572-b275c32991ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:51.884641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa42c37-bd8b-45a6-90b9-f4a038272527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:51.885463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112ba66-cbee-4ae2-9572-b275c32991ec service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:51.887052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa42c37-bd8b-45a6-90b9-f4a038272527 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:51.905589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 861/1720] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:51.906765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa42c37-bd8b-45a6-90b9-f4a038272527 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:51.927880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 860/1721] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:51 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:52.088499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:52.116771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa42c37-bd8b-45a6-90b9-f4a038272527 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:52.136038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d569d-856d-44a2-a9b1-30892d71e3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:52.139660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d569d-856d-44a2-a9b1-30892d71e3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:52.141610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d569d-856d-44a2-a9b1-30892d71e3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:52.141984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7d569d-856d-44a2-a9b1-30892d71e3a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:52.142728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d569d-856d-44a2-a9b1-30892d71e3a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:52.145920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7d569d-856d-44a2-a9b1-30892d71e3a9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:52.197466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 862/1722] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:51 2022] POST /identity/v3/users => generated 285 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:52.202722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112ba66-cbee-4ae2-9572-b275c32991ec service nova] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:52.223660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112ba66-cbee-4ae2-9572-b275c32991ec service nova] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:52.238652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b61823-167d-491d-8a02-ce8db57bd899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:52.239353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b61823-167d-491d-8a02-ce8db57bd899 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:52.239728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b61823-167d-491d-8a02-ce8db57bd899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:52.240227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b61823-167d-491d-8a02-ce8db57bd899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:52.240976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b61823-167d-491d-8a02-ce8db57bd899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:52.242686 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b61823-167d-491d-8a02-ce8db57bd899 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:52.269326 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d569d-856d-44a2-a9b1-30892d71e3a9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:52.277369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 861/1723] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:52.289043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7d569d-856d-44a2-a9b1-30892d71e3a9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:52.298511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a57118-0c59-45eb-a350-d34f3039e4f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:52.299521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a57118-0c59-45eb-a350-d34f3039e4f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:52.300060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a57118-0c59-45eb-a350-d34f3039e4f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:52.300513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a57118-0c59-45eb-a350-d34f3039e4f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:52.301517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a57118-0c59-45eb-a350-d34f3039e4f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:52.304762 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a57118-0c59-45eb-a350-d34f3039e4f7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:52.320147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 863/1724] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:13:52.324112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b61823-167d-491d-8a02-ce8db57bd899 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:52.342135 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b61823-167d-491d-8a02-ce8db57bd899 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:52.351413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e2c7c-c31a-40f7-883c-d16028eeaa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:52.353439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e2c7c-c31a-40f7-883c-d16028eeaa13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:52.354011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e2c7c-c31a-40f7-883c-d16028eeaa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:52.354489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452e2c7c-c31a-40f7-883c-d16028eeaa13 admin admin] PATH_INFO: `/v3/projects/5bdcd206063940b7afd726318f28da39/users/22fa7f813ed94f779986a7527306a35a/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:52.383660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e2c7c-c31a-40f7-883c-d16028eeaa13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bdcd206063940b7afd726318f28da39, user_id=22fa7f813ed94f779986a7527306a35a, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:52.396599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452e2c7c-c31a-40f7-883c-d16028eeaa13 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:52.427681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 862/1725] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:52 2022] PUT /identity/v3/projects/5bdcd206063940b7afd726318f28da39/users/22fa7f813ed94f779986a7527306a35a/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:52.431877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c59a-bd1c-4ca7-8f9a-05bea2208904 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:52.432171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c59a-bd1c-4ca7-8f9a-05bea2208904 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:52.432451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b31c59a-bd1c-4ca7-8f9a-05bea2208904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:52.506124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9b31c59a-bd1c-4ca7-8f9a-05bea2208904 None None] MFA Rules not processed for user `22fa7f813ed94f779986a7527306a35a`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:52.508758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9b31c59a-bd1c-4ca7-8f9a-05bea2208904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:52.641826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 864/1726] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:52.643376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e2c7c-c31a-40f7-883c-d16028eeaa13 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:52.824435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452e2c7c-c31a-40f7-883c-d16028eeaa13 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:52.833187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e365acb-963f-452a-af6e-395bbb44d5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:52.833894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e365acb-963f-452a-af6e-395bbb44d5e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:52.834180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e365acb-963f-452a-af6e-395bbb44d5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:52.834490 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e365acb-963f-452a-af6e-395bbb44d5e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:52.835373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e365acb-963f-452a-af6e-395bbb44d5e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:52.837513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e365acb-963f-452a-af6e-395bbb44d5e7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:52.850377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8220c7-8e72-45c5-b099-705f8a877eac None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:52.850678 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8220c7-8e72-45c5-b099-705f8a877eac None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:52.851249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8220c7-8e72-45c5-b099-705f8a877eac None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:52.853084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 865/1727] 10.222.0.26 () {54 vars in 912 bytes} [Sat Sep 10 14:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:13:52.860456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132f8ea-8b64-48c4-b384-b861201bb3af None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:52.860736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132f8ea-8b64-48c4-b384-b861201bb3af None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:52.861018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132f8ea-8b64-48c4-b384-b861201bb3af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:52.918294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d132f8ea-8b64-48c4-b384-b861201bb3af None None] MFA Rules not processed for user `12c71caf31354ec193e3e2da231f1e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:52.920423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d132f8ea-8b64-48c4-b384-b861201bb3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:52.933008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 863/1728] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:52 2022] POST /identity/v3/projects => generated 373 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:13:52.939804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e365acb-963f-452a-af6e-395bbb44d5e7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:53.164587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 866/1729] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:13:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:13:53.188266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e365acb-963f-452a-af6e-395bbb44d5e7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:53.191227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132f8ea-8b64-48c4-b384-b861201bb3af None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:53.193843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d132f8ea-8b64-48c4-b384-b861201bb3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:53.225490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132f8ea-8b64-48c4-b384-b861201bb3af None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:53.228846 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b4104-df48-4719-8234-20b05b0f2c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:53.233827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b4104-df48-4719-8234-20b05b0f2c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:53.235533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b4104-df48-4719-8234-20b05b0f2c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:53.236993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b4104-df48-4719-8234-20b05b0f2c08 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:53.239362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b4104-df48-4719-8234-20b05b0f2c08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:53.254992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b4104-df48-4719-8234-20b05b0f2c08 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:53.334712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 864/1730] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:52 2022] POST /identity/v3/users => generated 293 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:53.342311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b4104-df48-4719-8234-20b05b0f2c08 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:53.371218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:53.395411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b4104-df48-4719-8234-20b05b0f2c08 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:53.406984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d5eea-a1f5-4856-852e-6a976f544605 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:53.407652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d5eea-a1f5-4856-852e-6a976f544605 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:53.407926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d5eea-a1f5-4856-852e-6a976f544605 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:53.408194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813d5eea-a1f5-4856-852e-6a976f544605 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:53.416360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d5eea-a1f5-4856-852e-6a976f544605 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:53.417858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813d5eea-a1f5-4856-852e-6a976f544605 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:53.421317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faccac1-496b-4a41-bd89-43585e12e6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:53.429291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 867/1731] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:53.430161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d5eea-a1f5-4856-852e-6a976f544605 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:53.432437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-813d5eea-a1f5-4856-852e-6a976f544605 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:53.446356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faccac1-496b-4a41-bd89-43585e12e6cd admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:53.446647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faccac1-496b-4a41-bd89-43585e12e6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:53.446928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faccac1-496b-4a41-bd89-43585e12e6cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:53.451911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faccac1-496b-4a41-bd89-43585e12e6cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:53.460617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faccac1-496b-4a41-bd89-43585e12e6cd admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:53.467099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813d5eea-a1f5-4856-852e-6a976f544605 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:53.514655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:53.515592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:53.517005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:53.517492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] PATH_INFO: `/v3/users/c27c43059bf74810b9953ac43ba3d72a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:53.521925 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 865/1732] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:53.524066 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c27c43059bf74810b9953ac43ba3d72a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:53.524704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faccac1-496b-4a41-bd89-43585e12e6cd admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:53.529218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:53.558818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:53.563855 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faccac1-496b-4a41-bd89-43585e12e6cd admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:53.587501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2b625-adb6-4b20-b0b9-9c5bda97b9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:53.591433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2b625-adb6-4b20-b0b9-9c5bda97b9d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:53.592933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2b625-adb6-4b20-b0b9-9c5bda97b9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:53.595338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e2b625-adb6-4b20-b0b9-9c5bda97b9d3 admin admin] PATH_INFO: `/v3/projects/f0e39fe439a945f7a3fdc001ae1f9976/users/8d0b014191db4fe4a473ae508202f6cd/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:53.666498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2b625-adb6-4b20-b0b9-9c5bda97b9d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e39fe439a945f7a3fdc001ae1f9976, user_id=8d0b014191db4fe4a473ae508202f6cd, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:53.678588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e2b625-adb6-4b20-b0b9-9c5bda97b9d3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:53.709466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c27c43059bf74810b9953ac43ba3d72a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:53.728298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 866/1733] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:53 2022] PUT /identity/v3/projects/f0e39fe439a945f7a3fdc001ae1f9976/users/8d0b014191db4fe4a473ae508202f6cd/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:53.733646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2b625-adb6-4b20-b0b9-9c5bda97b9d3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:53.736027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c27c43059bf74810b9953ac43ba3d72a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:53.736725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c27c43059bf74810b9953ac43ba3d72a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:53.751040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c27c43059bf74810b9953ac43ba3d72a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:53.792170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2656d229-f767-43da-9a3c-188ec53488e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c27c43059bf74810b9953ac43ba3d72a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:53.901446 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 868/1734] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:13:53 2022] DELETE /identity/v3/users/c27c43059bf74810b9953ac43ba3d72a => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:53.906344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb69973-1b13-4435-9868-5b3606b575bf None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:53.906626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb69973-1b13-4435-9868-5b3606b575bf None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:53.906902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb69973-1b13-4435-9868-5b3606b575bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:53.982541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3eb69973-1b13-4435-9868-5b3606b575bf None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:53.985877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3eb69973-1b13-4435-9868-5b3606b575bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:53.989348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e2b625-adb6-4b20-b0b9-9c5bda97b9d3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:54.122293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 869/1735] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:54.147414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb69973-1b13-4435-9868-5b3606b575bf None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:54.149090 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3eb69973-1b13-4435-9868-5b3606b575bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:54.272545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d962919-2323-4b4f-bd60-2b488fb3330d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:54.273184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962919-2323-4b4f-bd60-2b488fb3330d admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:54.273460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962919-2323-4b4f-bd60-2b488fb3330d admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:54.273732 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d962919-2323-4b4f-bd60-2b488fb3330d admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:54.285474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d962919-2323-4b4f-bd60-2b488fb3330d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:54.287464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d962919-2323-4b4f-bd60-2b488fb3330d admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:54.317426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb69973-1b13-4435-9868-5b3606b575bf None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:54.340356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c463d-f552-4eaa-b05b-65d2343cacd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:54.341838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c463d-f552-4eaa-b05b-65d2343cacd8 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:54.342305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c463d-f552-4eaa-b05b-65d2343cacd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:54.342836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c463d-f552-4eaa-b05b-65d2343cacd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:54.343600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 867/1736] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 610 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:54.347014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d962919-2323-4b4f-bd60-2b488fb3330d admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:54.351061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0d962919-2323-4b4f-bd60-2b488fb3330d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:54.381682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d962919-2323-4b4f-bd60-2b488fb3330d admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:54.385996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c463d-f552-4eaa-b05b-65d2343cacd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:54.387292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c463d-f552-4eaa-b05b-65d2343cacd8 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:54.398656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22797b30-5d8f-46df-921b-caeeae8c70d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:54.401132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22797b30-5d8f-46df-921b-caeeae8c70d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:54.401424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22797b30-5d8f-46df-921b-caeeae8c70d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:54.401791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22797b30-5d8f-46df-921b-caeeae8c70d3 admin admin] PATH_INFO: `/v3/projects/f0e39fe439a945f7a3fdc001ae1f9976/users/8d0b014191db4fe4a473ae508202f6cd/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:54.406738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22797b30-5d8f-46df-921b-caeeae8c70d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0e39fe439a945f7a3fdc001ae1f9976, user_id=8d0b014191db4fe4a473ae508202f6cd, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:54.414203 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 870/1737] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:13:54.418758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22797b30-5d8f-46df-921b-caeeae8c70d3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:54.445551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 868/1738] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:54 2022] PUT /identity/v3/projects/f0e39fe439a945f7a3fdc001ae1f9976/users/8d0b014191db4fe4a473ae508202f6cd/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:54.456832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce0d46-4606-4754-b85c-a588e2625203 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:54.457225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce0d46-4606-4754-b85c-a588e2625203 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:54.457602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce0d46-4606-4754-b85c-a588e2625203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:54.529805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-93ce0d46-4606-4754-b85c-a588e2625203 None None] MFA Rules not processed for user `8d0b014191db4fe4a473ae508202f6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:54.531142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-93ce0d46-4606-4754-b85c-a588e2625203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:54.567855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:54.767400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 871/1739] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:54.779354 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22797b30-5d8f-46df-921b-caeeae8c70d3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:54.782681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-22797b30-5d8f-46df-921b-caeeae8c70d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:54.892981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22797b30-5d8f-46df-921b-caeeae8c70d3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:54.906272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162751d-8860-4df3-840c-3386106d8f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:54.906929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162751d-8860-4df3-840c-3386106d8f5d service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:54.907210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162751d-8860-4df3-840c-3386106d8f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:54.907513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162751d-8860-4df3-840c-3386106d8f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:54.929976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162751d-8860-4df3-840c-3386106d8f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:54.931728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162751d-8860-4df3-840c-3386106d8f5d service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:54.957721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:54.978109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 869/1740] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:13:55.118803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c4a1c-b81b-45d2-9db2-207e4e6bccf2 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:55.119767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c4a1c-b81b-45d2-9db2-207e4e6bccf2 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:55.120274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4c4a1c-b81b-45d2-9db2-207e4e6bccf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:55.191152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1d4c4a1c-b81b-45d2-9db2-207e4e6bccf2 None None] MFA Rules not processed for user `8ea8ba61dfa54cbc9a45543d20f31e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:55.192918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1d4c4a1c-b81b-45d2-9db2-207e4e6bccf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:55.194129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f518eb5-b4e6-4bbb-88eb-8c352205021d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:55.194500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f518eb5-b4e6-4bbb-88eb-8c352205021d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:55.194890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f518eb5-b4e6-4bbb-88eb-8c352205021d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:55.285305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-3f518eb5-b4e6-4bbb-88eb-8c352205021d None None] MFA Rules not processed for user `8f96c9a2a49047b5b8abd3b2c1475a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:55.290360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3f518eb5-b4e6-4bbb-88eb-8c352205021d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:55.428751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 872/1741] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:55.545749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 870/1742] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:56.069374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c4a1c-b81b-45d2-9db2-207e4e6bccf2 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:56.222519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4c4a1c-b81b-45d2-9db2-207e4e6bccf2 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:56.230222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:56.232582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:56.233264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:56.234268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] PATH_INFO: `/v3/users/22fa7f813ed94f779986a7527306a35a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:56.236437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22fa7f813ed94f779986a7527306a35a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:56.240286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:56.302438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f518eb5-b4e6-4bbb-88eb-8c352205021d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:56.306060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3f518eb5-b4e6-4bbb-88eb-8c352205021d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:56.334379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f518eb5-b4e6-4bbb-88eb-8c352205021d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:56.361636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22fa7f813ed94f779986a7527306a35a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.364889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:56.365557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:56.365831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:56.366100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] PATH_INFO: `/v3/projects/81a7db658c9f4e9babeab02e89b18946` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:56.367267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] RBAC: Authorizing `identity:delete_project(project_id=81a7db658c9f4e9babeab02e89b18946)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:56.369245 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:56.371475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22fa7f813ed94f779986a7527306a35a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.372502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22fa7f813ed94f779986a7527306a35a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.373448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22fa7f813ed94f779986a7527306a35a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.381124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:56.384580 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3216d09b-2eba-4457-a2d0-17fa6851c376 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22fa7f813ed94f779986a7527306a35a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.416713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 873/1743] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:13:56 2022] DELETE /identity/v3/users/22fa7f813ed94f779986a7527306a35a => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:56.424512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9e82f-ba0d-461d-9c1b-dd4d72bd2278 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:56.424794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9e82f-ba0d-461d-9c1b-dd4d72bd2278 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:56.425061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9e82f-ba0d-461d-9c1b-dd4d72bd2278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:56.506096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5de9e82f-ba0d-461d-9c1b-dd4d72bd2278 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:56.514876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5de9e82f-ba0d-461d-9c1b-dd4d72bd2278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:56.559679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81a7db658c9f4e9babeab02e89b18946', 'request_id': 'req-33e758f1-434b-4052-acd1-1b8f8242239a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.590367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81a7db658c9f4e9babeab02e89b18946', 'request_id': 'req-33e758f1-434b-4052-acd1-1b8f8242239a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.612140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81a7db658c9f4e9babeab02e89b18946', 'request_id': 'req-33e758f1-434b-4052-acd1-1b8f8242239a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.652979 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] The token cache is being invalidate because project 81a7db658c9f4e9babeab02e89b18946 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:13:56.654261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.657330 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.664764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 871/1744] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:13:56 2022] DELETE /identity/v3/projects/81a7db658c9f4e9babeab02e89b18946 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:56.675311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:56.681325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:56.712982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 874/1745] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:56.733113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9e82f-ba0d-461d-9c1b-dd4d72bd2278 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:56.734694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5de9e82f-ba0d-461d-9c1b-dd4d72bd2278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:56.853749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e758f1-434b-4052-acd1-1b8f8242239a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:56.870145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:56.871266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:56.874690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:56.876357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] PATH_INFO: `/v3/users/4ce12bd80b7444dcb6ed90d63157021e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:56.878039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ce12bd80b7444dcb6ed90d63157021e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:56.879226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9e82f-ba0d-461d-9c1b-dd4d72bd2278 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:56.880578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:56.884898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff56dc-dd08-4616-adfe-d1e9c737e93a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:56.886010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff56dc-dd08-4616-adfe-d1e9c737e93a service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:56.886467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff56dc-dd08-4616-adfe-d1e9c737e93a service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:56.886880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff56dc-dd08-4616-adfe-d1e9c737e93a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:56.897925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff56dc-dd08-4616-adfe-d1e9c737e93a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:56.899899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ff56dc-dd08-4616-adfe-d1e9c737e93a service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:56.922726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 875/1746] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:13:56.949567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:56.952043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654cf0f-0193-4683-8e53-ad9ed88109f8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:56.952435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654cf0f-0193-4683-8e53-ad9ed88109f8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:56.952807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654cf0f-0193-4683-8e53-ad9ed88109f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:56.975502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ce12bd80b7444dcb6ed90d63157021e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.975502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ce12bd80b7444dcb6ed90d63157021e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:56.988476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ce12bd80b7444dcb6ed90d63157021e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:57.002366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e654cf0f-0193-4683-8e53-ad9ed88109f8 None None] MFA Rules not processed for user `810f0fa1f7774b37a09a91964c40b823`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:57.003825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e654cf0f-0193-4683-8e53-ad9ed88109f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:57.005350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ce12bd80b7444dcb6ed90d63157021e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:57.006229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ce12bd80b7444dcb6ed90d63157021e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:57.030822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 872/1747] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:13:56 2022] DELETE /identity/v3/users/4ce12bd80b7444dcb6ed90d63157021e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:57.155131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 876/1748] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:56 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:57.886382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:57.889288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:58.039201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654cf0f-0193-4683-8e53-ad9ed88109f8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:58.048489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e654cf0f-0193-4683-8e53-ad9ed88109f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:58.085792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654cf0f-0193-4683-8e53-ad9ed88109f8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:58.091812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b133099-86a2-4066-8a2f-be6d47ed8d2b admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:58.100753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:58.101655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:58.101940 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:58.102208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] PATH_INFO: `/v3/projects/d00b655c92d64645a3f46871849e4c4e` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:58.103250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:58.104427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:58.105754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:58.106321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] PATH_INFO: `/v3/projects/5bdcd206063940b7afd726318f28da39` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:58.106510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d00b655c92d64645a3f46871849e4c4e)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:58.107739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bdcd206063940b7afd726318f28da39)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:58.110985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:58.112753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:58.175290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:58.235071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bdcd206063940b7afd726318f28da39', 'request_id': 'req-f56d9f96-28ce-41c3-af0b-851c460cc6c6'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.249389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bdcd206063940b7afd726318f28da39', 'request_id': 'req-f56d9f96-28ce-41c3-af0b-851c460cc6c6'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.257632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bdcd206063940b7afd726318f28da39', 'request_id': 'req-f56d9f96-28ce-41c3-af0b-851c460cc6c6'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.273267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd00b655c92d64645a3f46871849e4c4e', 'request_id': 'req-7156d174-235d-4d54-892f-4f61825f13f4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.275267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] The token cache is being invalidate because project 5bdcd206063940b7afd726318f28da39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:13:58.275665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.276613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.279771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 873/1749] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:13:57 2022] DELETE /identity/v3/projects/5bdcd206063940b7afd726318f28da39 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:58.283433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:58.285389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:58.294996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd00b655c92d64645a3f46871849e4c4e', 'request_id': 'req-7156d174-235d-4d54-892f-4f61825f13f4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.316938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd00b655c92d64645a3f46871849e4c4e', 'request_id': 'req-7156d174-235d-4d54-892f-4f61825f13f4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.371086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] The token cache is being invalidate because project d00b655c92d64645a3f46871849e4c4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:13:58.371318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.374733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7156d174-235d-4d54-892f-4f61825f13f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.384888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 877/1750] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:13:58 2022] DELETE /identity/v3/projects/d00b655c92d64645a3f46871849e4c4e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:58.386765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfb6fb-7fc5-4a3f-bb97-e0157a1f60fe None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:58.392928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfb6fb-7fc5-4a3f-bb97-e0157a1f60fe None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:58.395498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfb6fb-7fc5-4a3f-bb97-e0157a1f60fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:58.419531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:58.470542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d9f96-28ce-41c3-af0b-851c460cc6c6 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:58.484519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-cecfb6fb-7fc5-4a3f-bb97-e0157a1f60fe None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:58.489534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cecfb6fb-7fc5-4a3f-bb97-e0157a1f60fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:58.544055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:58.545002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:58.545338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:58.545619 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] PATH_INFO: `/v3/users/8d0b014191db4fe4a473ae508202f6cd` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:58.547460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d0b014191db4fe4a473ae508202f6cd)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:58.549459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:58.650084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 878/1751] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:58.656872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d0b014191db4fe4a473ae508202f6cd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.659251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d0b014191db4fe4a473ae508202f6cd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.659390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eabc1-b4b9-4966-bcbb-1f9bdbc79a6f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:58.659746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eabc1-b4b9-4966-bcbb-1f9bdbc79a6f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:58.659927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928eabc1-b4b9-4966-bcbb-1f9bdbc79a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:58.673921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d0b014191db4fe4a473ae508202f6cd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.689819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d0b014191db4fe4a473ae508202f6cd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.692187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d0b014191db4fe4a473ae508202f6cd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:58.719487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-928eabc1-b4b9-4966-bcbb-1f9bdbc79a6f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:58.724805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-928eabc1-b4b9-4966-bcbb-1f9bdbc79a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:58.758605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 879/1752] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:13:58.764131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eabc1-b4b9-4966-bcbb-1f9bdbc79a6f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:58.768967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 874/1753] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:13:58 2022] DELETE /identity/v3/users/8d0b014191db4fe4a473ae508202f6cd => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:58.795982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928eabc1-b4b9-4966-bcbb-1f9bdbc79a6f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:58.967942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2745c-ddd4-47ef-94ea-59e55fb3f3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:58.968660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2745c-ddd4-47ef-94ea-59e55fb3f3bd admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:58.968943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2745c-ddd4-47ef-94ea-59e55fb3f3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:58.969216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e2745c-ddd4-47ef-94ea-59e55fb3f3bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:58.970221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2745c-ddd4-47ef-94ea-59e55fb3f3bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:13:58.970341 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2745c-ddd4-47ef-94ea-59e55fb3f3bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:58.972047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e2745c-ddd4-47ef-94ea-59e55fb3f3bd admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:58.992415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 880/1754] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:13:58.993607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:59.011295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeff555-77ce-4507-a928-4ba22a02ce75 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:59.023253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c9cc0-1128-4e04-b78a-25eb903f8d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:59.024158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c9cc0-1128-4e04-b78a-25eb903f8d84 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:59.027654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c9cc0-1128-4e04-b78a-25eb903f8d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:59.028151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c9cc0-1128-4e04-b78a-25eb903f8d84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:59.029181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c9cc0-1128-4e04-b78a-25eb903f8d84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:59.031342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c9cc0-1128-4e04-b78a-25eb903f8d84 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:59.099898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 875/1755] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:13:58 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:59.102343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2745c-ddd4-47ef-94ea-59e55fb3f3bd admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:59.119251 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e2745c-ddd4-47ef-94ea-59e55fb3f3bd admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:59.236755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:59.314823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5ceda-4d0d-4464-8713-9b72cab803ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:59.315487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5ceda-4d0d-4464-8713-9b72cab803ea admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:59.318043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5ceda-4d0d-4464-8713-9b72cab803ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:59.318334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a5ceda-4d0d-4464-8713-9b72cab803ea admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:59.319008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5ceda-4d0d-4464-8713-9b72cab803ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:59.344620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a5ceda-4d0d-4464-8713-9b72cab803ea admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:59.379928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 881/1756] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:13:59 2022] POST /identity/v3/users => generated 293 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:13:59.381109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c9cc0-1128-4e04-b78a-25eb903f8d84 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:59.410831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c9cc0-1128-4e04-b78a-25eb903f8d84 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:59.447457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865441e-90e5-4af8-8f60-d1ad8e8099d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:59.448108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865441e-90e5-4af8-8f60-d1ad8e8099d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:59.448384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865441e-90e5-4af8-8f60-d1ad8e8099d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:59.448657 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865441e-90e5-4af8-8f60-d1ad8e8099d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:59.460484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865441e-90e5-4af8-8f60-d1ad8e8099d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:59.462409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865441e-90e5-4af8-8f60-d1ad8e8099d5 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:59.505889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5ceda-4d0d-4464-8713-9b72cab803ea admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:59.508303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-78a5ceda-4d0d-4464-8713-9b72cab803ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:59.534569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 876/1757] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:59.537215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865441e-90e5-4af8-8f60-d1ad8e8099d5 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:59.549105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a5ceda-4d0d-4464-8713-9b72cab803ea admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:59.574390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:59.575249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:59.575656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:59.575996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] PATH_INFO: `/v3/projects/f0e39fe439a945f7a3fdc001ae1f9976` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:59.580078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0e39fe439a945f7a3fdc001ae1f9976)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:59.587401 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:59.590336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865441e-90e5-4af8-8f60-d1ad8e8099d5 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:59.607975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f317-f4de-4f5e-8534-9dd56c0b270a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:59.609173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f317-f4de-4f5e-8534-9dd56c0b270a admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:59.610118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f317-f4de-4f5e-8534-9dd56c0b270a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:59.610572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7536f317-f4de-4f5e-8534-9dd56c0b270a admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:59.611472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536f317-f4de-4f5e-8534-9dd56c0b270a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:59.616687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7536f317-f4de-4f5e-8534-9dd56c0b270a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:59.645315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 877/1758] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:13:59.652965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f317-f4de-4f5e-8534-9dd56c0b270a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:13:59.666849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7536f317-f4de-4f5e-8534-9dd56c0b270a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:13:59.679178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295acfda-4e7a-4849-aab8-4e32d6d5d00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:13:59.680537 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295acfda-4e7a-4849-aab8-4e32d6d5d00c admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:59.681036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295acfda-4e7a-4849-aab8-4e32d6d5d00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:59.681317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295acfda-4e7a-4849-aab8-4e32d6d5d00c admin admin] PATH_INFO: `/v3/projects/7ca8d0f1e9224b28b681d9c9c53a46ef/users/ecd7933e7dda4722bac2d05fb048bf59/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:59.712123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295acfda-4e7a-4849-aab8-4e32d6d5d00c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca8d0f1e9224b28b681d9c9c53a46ef, user_id=ecd7933e7dda4722bac2d05fb048bf59, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:13:59.722159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295acfda-4e7a-4849-aab8-4e32d6d5d00c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:13:59.736796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0e39fe439a945f7a3fdc001ae1f9976', 'request_id': 'req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:59.740984 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:13:59.744165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0e39fe439a945f7a3fdc001ae1f9976', 'request_id': 'req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:59.770698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 878/1759] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:13:59 2022] PUT /identity/v3/projects/7ca8d0f1e9224b28b681d9c9c53a46ef/users/ecd7933e7dda4722bac2d05fb048bf59/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:13:59.774361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171dfe2-2e3f-4732-ba63-6522b418469b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:59.775588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171dfe2-2e3f-4732-ba63-6522b418469b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:59.775884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f171dfe2-2e3f-4732-ba63-6522b418469b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:59.780362 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0e39fe439a945f7a3fdc001ae1f9976', 'request_id': 'req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:59.814651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] The token cache is being invalidate because project f0e39fe439a945f7a3fdc001ae1f9976 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:13:59.815959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:59.820545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2a30b2b2-f4cb-4110-ae53-8cbb061f15f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:13:59.853634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 882/1760] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:13:59 2022] DELETE /identity/v3/projects/f0e39fe439a945f7a3fdc001ae1f9976 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:13:59.859285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84479e-8d90-4951-af33-deac03110fb0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:13:59.860615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84479e-8d90-4951-af33-deac03110fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:13:59.862077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84479e-8d90-4951-af33-deac03110fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:13:59.882808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-f171dfe2-2e3f-4732-ba63-6522b418469b None None] MFA Rules not processed for user `ecd7933e7dda4722bac2d05fb048bf59`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:59.888579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f171dfe2-2e3f-4732-ba63-6522b418469b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:13:59.957414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7f84479e-8d90-4951-af33-deac03110fb0 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:13:59.960406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7f84479e-8d90-4951-af33-deac03110fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:00.138225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 879/1761] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:00.141466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 883/1762] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:00.145560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171dfe2-2e3f-4732-ba63-6522b418469b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:00.148394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f171dfe2-2e3f-4732-ba63-6522b418469b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:00.160475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066bb6-4bc3-4d98-9066-5001f695f566 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:00.160798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066bb6-4bc3-4d98-9066-5001f695f566 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:00.161070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b066bb6-4bc3-4d98-9066-5001f695f566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:00.201034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f171dfe2-2e3f-4732-ba63-6522b418469b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:00.214207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46873d15-05f2-4bc4-b5b0-a9f2eb242182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:00.214868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46873d15-05f2-4bc4-b5b0-a9f2eb242182 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:00.217221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46873d15-05f2-4bc4-b5b0-a9f2eb242182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:00.218712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46873d15-05f2-4bc4-b5b0-a9f2eb242182 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:00.219725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46873d15-05f2-4bc4-b5b0-a9f2eb242182 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:00.225151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46873d15-05f2-4bc4-b5b0-a9f2eb242182 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:00.229854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4b066bb6-4bc3-4d98-9066-5001f695f566 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:00.234388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4b066bb6-4bc3-4d98-9066-5001f695f566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:00.256443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 884/1763] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:00.260922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b066bb6-4bc3-4d98-9066-5001f695f566 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:00.298365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b066bb6-4bc3-4d98-9066-5001f695f566 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:00.312897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 880/1764] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:00 2022] POST /identity/v3/projects => generated 385 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:00.313237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f899d3c-0e9c-4e45-9b16-24ae7f49f32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:00.313933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f899d3c-0e9c-4e45-9b16-24ae7f49f32d admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:00.314201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f899d3c-0e9c-4e45-9b16-24ae7f49f32d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:00.314469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f899d3c-0e9c-4e45-9b16-24ae7f49f32d admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:00.315127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f899d3c-0e9c-4e45-9b16-24ae7f49f32d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:00.315422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f899d3c-0e9c-4e45-9b16-24ae7f49f32d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:00.317055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46873d15-05f2-4bc4-b5b0-a9f2eb242182 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:00.327427 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f899d3c-0e9c-4e45-9b16-24ae7f49f32d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:00.374918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 885/1765] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:00.376349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f899d3c-0e9c-4e45-9b16-24ae7f49f32d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:00.404873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f899d3c-0e9c-4e45-9b16-24ae7f49f32d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:00.482246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:00.548433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46873d15-05f2-4bc4-b5b0-a9f2eb242182 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:00.553273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37015a22-1cd3-431d-8256-78f15eacc8ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:00.553909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37015a22-1cd3-431d-8256-78f15eacc8ac admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:00.554185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37015a22-1cd3-431d-8256-78f15eacc8ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:00.554457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37015a22-1cd3-431d-8256-78f15eacc8ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:00.555100 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37015a22-1cd3-431d-8256-78f15eacc8ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:00.557284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37015a22-1cd3-431d-8256-78f15eacc8ac admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:00.561918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8cca4-8627-4c59-b873-fdabf5561168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:00.562998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8cca4-8627-4c59-b873-fdabf5561168 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:00.563497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8cca4-8627-4c59-b873-fdabf5561168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:00.563944 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f8cca4-8627-4c59-b873-fdabf5561168 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:00.564930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8cca4-8627-4c59-b873-fdabf5561168 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:00.582243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f8cca4-8627-4c59-b873-fdabf5561168 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:00.643094 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 881/1766] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:00 2022] POST /identity/v3/users => generated 299 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:00.647410 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8cca4-8627-4c59-b873-fdabf5561168 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:00.669739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 886/1767] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:00 2022] POST /identity/v3/projects => generated 365 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:00.676271 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37015a22-1cd3-431d-8256-78f15eacc8ac admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:00.687383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f8cca4-8627-4c59-b873-fdabf5561168 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:00.783058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:00.805868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093d2b7-40df-44d8-b028-a11fe305dacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:00.810213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093d2b7-40df-44d8-b028-a11fe305dacb admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:00.810850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093d2b7-40df-44d8-b028-a11fe305dacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:00.811228 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093d2b7-40df-44d8-b028-a11fe305dacb admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:00.823478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093d2b7-40df-44d8-b028-a11fe305dacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:00.823478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093d2b7-40df-44d8-b028-a11fe305dacb admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:00.832083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 882/1768] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:00.838824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093d2b7-40df-44d8-b028-a11fe305dacb admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:00.869196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093d2b7-40df-44d8-b028-a11fe305dacb admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:00.885258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892763c3-ff7d-4c6e-ab3e-360f1bed9cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:00.885948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892763c3-ff7d-4c6e-ab3e-360f1bed9cae admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:00.886228 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892763c3-ff7d-4c6e-ab3e-360f1bed9cae admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:00.886505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892763c3-ff7d-4c6e-ab3e-360f1bed9cae admin admin] PATH_INFO: `/v3/projects/4d0c66d54f974e528e988a4f588725ab/users/4fca43765baf4cdbb3c17c3dd7651db8/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:00.928409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37015a22-1cd3-431d-8256-78f15eacc8ac admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:00.941377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892763c3-ff7d-4c6e-ab3e-360f1bed9cae admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0c66d54f974e528e988a4f588725ab, user_id=4fca43765baf4cdbb3c17c3dd7651db8, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:00.947298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892763c3-ff7d-4c6e-ab3e-360f1bed9cae admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:00.949423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2528c-b085-4eff-a6d1-cbf48745d46b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:00.953584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2528c-b085-4eff-a6d1-cbf48745d46b admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:00.954034 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2528c-b085-4eff-a6d1-cbf48745d46b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:00.963651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d2528c-b085-4eff-a6d1-cbf48745d46b admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:00.964352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2528c-b085-4eff-a6d1-cbf48745d46b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:00.977054 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d2528c-b085-4eff-a6d1-cbf48745d46b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:00.988645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 883/1769] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:00 2022] PUT /identity/v3/projects/4d0c66d54f974e528e988a4f588725ab/users/4fca43765baf4cdbb3c17c3dd7651db8/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:00.998745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892763c3-ff7d-4c6e-ab3e-360f1bed9cae admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:01.027777 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 887/1770] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:00 2022] POST /identity/v3/users => generated 283 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:01.032776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2528c-b085-4eff-a6d1-cbf48745d46b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:01.054606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d2528c-b085-4eff-a6d1-cbf48745d46b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:01.130490 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892763c3-ff7d-4c6e-ab3e-360f1bed9cae admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:01.136233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990f94f-8c12-4047-920b-27b9f6295fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:01.137239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990f94f-8c12-4047-920b-27b9f6295fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:01.137644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990f94f-8c12-4047-920b-27b9f6295fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:01.138078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990f94f-8c12-4047-920b-27b9f6295fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:01.139407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990f94f-8c12-4047-920b-27b9f6295fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:01.141139 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990f94f-8c12-4047-920b-27b9f6295fe6 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:01.163872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d3e7b-4c92-40f3-8ac3-85f011018973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:01.166034 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d3e7b-4c92-40f3-8ac3-85f011018973 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:01.166393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d3e7b-4c92-40f3-8ac3-85f011018973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:01.166833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d3e7b-4c92-40f3-8ac3-85f011018973 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:01.167305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 884/1771] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:01.167937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d3e7b-4c92-40f3-8ac3-85f011018973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:01.171856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990f94f-8c12-4047-920b-27b9f6295fe6 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:01.172467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d3e7b-4c92-40f3-8ac3-85f011018973 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:01.194481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990f94f-8c12-4047-920b-27b9f6295fe6 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:01.203369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 888/1772] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:01.209512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d3e7b-4c92-40f3-8ac3-85f011018973 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:01.218233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43975657-c98a-455c-b74d-ef1e9424b0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:01.219143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43975657-c98a-455c-b74d-ef1e9424b0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:01.219594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43975657-c98a-455c-b74d-ef1e9424b0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:01.219969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43975657-c98a-455c-b74d-ef1e9424b0a9 admin admin] PATH_INFO: `/v3/projects/4d0c66d54f974e528e988a4f588725ab/users/4fca43765baf4cdbb3c17c3dd7651db8/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:01.222205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43975657-c98a-455c-b74d-ef1e9424b0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0c66d54f974e528e988a4f588725ab, user_id=4fca43765baf4cdbb3c17c3dd7651db8, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:01.225281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43975657-c98a-455c-b74d-ef1e9424b0a9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:01.236109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d3e7b-4c92-40f3-8ac3-85f011018973 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:01.238954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 885/1773] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:01 2022] PUT /identity/v3/projects/4d0c66d54f974e528e988a4f588725ab/users/4fca43765baf4cdbb3c17c3dd7651db8/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:01.251155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b169a-b476-4681-9553-c401c4b704b6 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:01.251363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b169a-b476-4681-9553-c401c4b704b6 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:01.251785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b169a-b476-4681-9553-c401c4b704b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:01.316565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:01.343069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-832b169a-b476-4681-9553-c401c4b704b6 None None] MFA Rules not processed for user `4fca43765baf4cdbb3c17c3dd7651db8`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:01.344471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-832b169a-b476-4681-9553-c401c4b704b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:01.353524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5091a-ed86-4514-843c-4ecab0cb0ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:01.354885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5091a-ed86-4514-843c-4ecab0cb0ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:01.357457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5091a-ed86-4514-843c-4ecab0cb0ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:01.357747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f5091a-ed86-4514-843c-4ecab0cb0ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:01.358553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5091a-ed86-4514-843c-4ecab0cb0ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:01.360253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f5091a-ed86-4514-843c-4ecab0cb0ec4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:01.379381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 889/1774] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:01.381939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5091a-ed86-4514-843c-4ecab0cb0ec4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:01.405649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f5091a-ed86-4514-843c-4ecab0cb0ec4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:01.427586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bbbc0-ad59-4315-a419-311b9867e943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:01.427586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bbbc0-ad59-4315-a419-311b9867e943 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:01.427586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bbbc0-ad59-4315-a419-311b9867e943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:01.427586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bbbc0-ad59-4315-a419-311b9867e943 admin admin] PATH_INFO: `/v3/projects/105a7d6b44974874aeda9e0fe2b5f112/users/f309030243944da6b933fd536a816591/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:01.448970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bbbc0-ad59-4315-a419-311b9867e943 admin admin] RBAC: Authorizing `identity:create_grant(project_id=105a7d6b44974874aeda9e0fe2b5f112, user_id=f309030243944da6b933fd536a816591, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:01.453493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bbbc0-ad59-4315-a419-311b9867e943 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:01.474355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 890/1775] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:01 2022] PUT /identity/v3/projects/105a7d6b44974874aeda9e0fe2b5f112/users/f309030243944da6b933fd536a816591/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:01.478451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b032d-7d05-425a-9200-9f894c53099f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:01.478451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b032d-7d05-425a-9200-9f894c53099f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:01.478451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0b032d-7d05-425a-9200-9f894c53099f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:01.516641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 886/1776] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:01.517927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b42250-578b-4bbd-acdf-eceff592c11d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:01.518208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b42250-578b-4bbd-acdf-eceff592c11d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:01.518498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b42250-578b-4bbd-acdf-eceff592c11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:01.570475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5a0b032d-7d05-425a-9200-9f894c53099f None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:01.573532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5a0b032d-7d05-425a-9200-9f894c53099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:01.584280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-03b42250-578b-4bbd-acdf-eceff592c11d None None] MFA Rules not processed for user `f309030243944da6b933fd536a816591`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:01.587259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-03b42250-578b-4bbd-acdf-eceff592c11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:01.637037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:01.749028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 891/1777] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:14:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:01.749909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b032d-7d05-425a-9200-9f894c53099f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:01.752711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5a0b032d-7d05-425a-9200-9f894c53099f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:01.798774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 887/1778] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:01.799149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b42250-578b-4bbd-acdf-eceff592c11d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:01.955046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0b032d-7d05-425a-9200-9f894c53099f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:01.965371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bee08-624f-4d30-90b0-a71e91d04dcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:01.966937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bee08-624f-4d30-90b0-a71e91d04dcc service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:01.968091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bee08-624f-4d30-90b0-a71e91d04dcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:01.968751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602bee08-624f-4d30-90b0-a71e91d04dcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:02.021807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b42250-578b-4bbd-acdf-eceff592c11d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:02.036739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4e6ef-7e51-4e7f-b96c-5e93776d05b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:02.038332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4e6ef-7e51-4e7f-b96c-5e93776d05b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:02.040698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4e6ef-7e51-4e7f-b96c-5e93776d05b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:02.041199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a4e6ef-7e51-4e7f-b96c-5e93776d05b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:02.048241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4e6ef-7e51-4e7f-b96c-5e93776d05b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:02.055091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a4e6ef-7e51-4e7f-b96c-5e93776d05b9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:02.114094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:02.163131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 888/1779] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:01 2022] POST /identity/v3/projects => generated 363 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:02.163131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4e6ef-7e51-4e7f-b96c-5e93776d05b9 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:02.165087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-14a4e6ef-7e51-4e7f-b96c-5e93776d05b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:02.184365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bee08-624f-4d30-90b0-a71e91d04dcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:02.186862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602bee08-624f-4d30-90b0-a71e91d04dcc service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:02.209929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 892/1780] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:02.210880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bee08-624f-4d30-90b0-a71e91d04dcc service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:02.241804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602bee08-624f-4d30-90b0-a71e91d04dcc service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:02.411705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ceb49-0909-4802-9027-913d1a4fc66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:02.415061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ceb49-0909-4802-9027-913d1a4fc66d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:02.415565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ceb49-0909-4802-9027-913d1a4fc66d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:02.415848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622ceb49-0909-4802-9027-913d1a4fc66d admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:02.416502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ceb49-0909-4802-9027-913d1a4fc66d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:02.420806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622ceb49-0909-4802-9027-913d1a4fc66d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:02.451766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a4e6ef-7e51-4e7f-b96c-5e93776d05b9 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:02.458143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2a286-0205-4109-97b8-51e90323ea3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:02.458774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2a286-0205-4109-97b8-51e90323ea3d service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:02.459118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2a286-0205-4109-97b8-51e90323ea3d service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:02.459319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a2a286-0205-4109-97b8-51e90323ea3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:02.491899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 893/1781] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:02 2022] POST /identity/v3/users => generated 288 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:02.491899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ceb49-0909-4802-9027-913d1a4fc66d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:02.510531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:02.515840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622ceb49-0909-4802-9027-913d1a4fc66d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:02.529233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:02.530032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:02.530313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:02.530583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] PATH_INFO: `/v3/users/ecd7933e7dda4722bac2d05fb048bf59` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:02.534693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecd7933e7dda4722bac2d05fb048bf59)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:02.538006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:02.551686 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2a286-0205-4109-97b8-51e90323ea3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:02.554273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a2a286-0205-4109-97b8-51e90323ea3d service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:02.567223 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 889/1782] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:14:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:14:02.571634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2a286-0205-4109-97b8-51e90323ea3d service swift] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:02.587285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a2a286-0205-4109-97b8-51e90323ea3d service swift] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:02.594533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7222c-3a7c-4f76-a9f5-5642c2ac6dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:02.595162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7222c-3a7c-4f76-a9f5-5642c2ac6dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:02.595471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7222c-3a7c-4f76-a9f5-5642c2ac6dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:02.595895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d7222c-3a7c-4f76-a9f5-5642c2ac6dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:02.596502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7222c-3a7c-4f76-a9f5-5642c2ac6dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:02.598629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d7222c-3a7c-4f76-a9f5-5642c2ac6dd5 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:02.615052 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 890/1783] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:02.621147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7222c-3a7c-4f76-a9f5-5642c2ac6dd5 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:02.648125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d7222c-3a7c-4f76-a9f5-5642c2ac6dd5 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:02.655218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e70849-39c2-4c2b-b35c-f7ef3ae18379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:02.656267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e70849-39c2-4c2b-b35c-f7ef3ae18379 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:02.656553 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e70849-39c2-4c2b-b35c-f7ef3ae18379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:02.656831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e70849-39c2-4c2b-b35c-f7ef3ae18379 admin admin] PATH_INFO: `/v3/projects/f802221799194033be2644ccf5198f9b/users/cfb18830a827477b9ad7353450191c15/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:02.683305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecd7933e7dda4722bac2d05fb048bf59'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:02.689826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e70849-39c2-4c2b-b35c-f7ef3ae18379 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f802221799194033be2644ccf5198f9b, user_id=cfb18830a827477b9ad7353450191c15, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:02.696332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e70849-39c2-4c2b-b35c-f7ef3ae18379 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:02.701973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecd7933e7dda4722bac2d05fb048bf59'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:02.704330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecd7933e7dda4722bac2d05fb048bf59'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:02.711796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecd7933e7dda4722bac2d05fb048bf59'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:02.733131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e1b72f0d-3cb7-4f2d-be94-122edfa5491b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecd7933e7dda4722bac2d05fb048bf59'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:02.755133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 891/1784] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:02 2022] PUT /identity/v3/projects/f802221799194033be2644ccf5198f9b/users/cfb18830a827477b9ad7353450191c15/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:02.758872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e70849-39c2-4c2b-b35c-f7ef3ae18379 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:02.763940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-02e70849-39c2-4c2b-b35c-f7ef3ae18379 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:02.859812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 894/1785] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:02 2022] DELETE /identity/v3/users/ecd7933e7dda4722bac2d05fb048bf59 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:02.863924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7794db2-579d-442c-9dc7-51263465c60c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:02.864246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7794db2-579d-442c-9dc7-51263465c60c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:02.868157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7794db2-579d-442c-9dc7-51263465c60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:02.954785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f7794db2-579d-442c-9dc7-51263465c60c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:02.956139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f7794db2-579d-442c-9dc7-51263465c60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:03.039181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:03.152056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 895/1786] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:03.156756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e70849-39c2-4c2b-b35c-f7ef3ae18379 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:03.169603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb48b50-eed3-4102-b692-560983aabc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:03.170482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb48b50-eed3-4102-b692-560983aabc21 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:03.170867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb48b50-eed3-4102-b692-560983aabc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:03.171860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb48b50-eed3-4102-b692-560983aabc21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:03.173086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb48b50-eed3-4102-b692-560983aabc21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:03.197405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7794db2-579d-442c-9dc7-51263465c60c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:03.199826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f7794db2-579d-442c-9dc7-51263465c60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:03.219151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb48b50-eed3-4102-b692-560983aabc21 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:03.267947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 892/1787] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 512 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:03.270055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb48b50-eed3-4102-b692-560983aabc21 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:03.297101 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb48b50-eed3-4102-b692-560983aabc21 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:03.316427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8875473-4246-454c-97e8-ae8b9f8b7861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:03.317287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8875473-4246-454c-97e8-ae8b9f8b7861 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:03.318274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8875473-4246-454c-97e8-ae8b9f8b7861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:03.319582 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8875473-4246-454c-97e8-ae8b9f8b7861 admin admin] PATH_INFO: `/v3/projects/f802221799194033be2644ccf5198f9b/users/cfb18830a827477b9ad7353450191c15/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:03.325521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8875473-4246-454c-97e8-ae8b9f8b7861 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f802221799194033be2644ccf5198f9b, user_id=cfb18830a827477b9ad7353450191c15, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:03.335314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8875473-4246-454c-97e8-ae8b9f8b7861 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:03.383889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 893/1788] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:03 2022] PUT /identity/v3/projects/f802221799194033be2644ccf5198f9b/users/cfb18830a827477b9ad7353450191c15/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:03.386153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff253122-1f22-4bd3-ba00-da210e60de0e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:03.386448 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff253122-1f22-4bd3-ba00-da210e60de0e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:03.386726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff253122-1f22-4bd3-ba00-da210e60de0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:03.495935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-ff253122-1f22-4bd3-ba00-da210e60de0e None None] MFA Rules not processed for user `cfb18830a827477b9ad7353450191c15`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:03.501031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ff253122-1f22-4bd3-ba00-da210e60de0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:03.620453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7794db2-579d-442c-9dc7-51263465c60c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:03.783120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9ee25-a481-4733-ac4e-36a9b8269a95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:03.815879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9ee25-a481-4733-ac4e-36a9b8269a95 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:03.817029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9ee25-a481-4733-ac4e-36a9b8269a95 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:03.820235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9ee25-a481-4733-ac4e-36a9b8269a95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:04.002711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 894/1789] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:03 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 618 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:04.015252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff253122-1f22-4bd3-ba00-da210e60de0e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:04.023505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ff253122-1f22-4bd3-ba00-da210e60de0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:04.041376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:04.078061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9ee25-a481-4733-ac4e-36a9b8269a95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:04.079772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9ee25-a481-4733-ac4e-36a9b8269a95 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:04.105927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 896/1790] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 909 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:04.218948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff253122-1f22-4bd3-ba00-da210e60de0e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:04.241157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863cead-120a-4889-afd9-fed5a913fad5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:04.242154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863cead-120a-4889-afd9-fed5a913fad5 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:04.245830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863cead-120a-4889-afd9-fed5a913fad5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:04.246295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863cead-120a-4889-afd9-fed5a913fad5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:04.282387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863cead-120a-4889-afd9-fed5a913fad5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:04.296635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863cead-120a-4889-afd9-fed5a913fad5 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:04.349955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 895/1791] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:04.572036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9ee25-a481-4733-ac4e-36a9b8269a95 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:04.591787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9ee25-a481-4733-ac4e-36a9b8269a95 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:04.611832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7301cb-1f8f-4223-af56-724403f4efca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:04.612475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7301cb-1f8f-4223-af56-724403f4efca service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:04.612754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7301cb-1f8f-4223-af56-724403f4efca service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:04.613031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7301cb-1f8f-4223-af56-724403f4efca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:04.623382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1b7301cb-1f8f-4223-af56-724403f4efca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:04.828758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:04.884618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7301cb-1f8f-4223-af56-724403f4efca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:04.887356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7301cb-1f8f-4223-af56-724403f4efca service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:04.932263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 897/1792] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:05.148496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255aaaa-3c84-4c46-8634-19541e735c1a None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:05.149061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255aaaa-3c84-4c46-8634-19541e735c1a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:05.149518 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6255aaaa-3c84-4c46-8634-19541e735c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:05.214422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-6255aaaa-3c84-4c46-8634-19541e735c1a None None] MFA Rules not processed for user `12c71caf31354ec193e3e2da231f1e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:05.221292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6255aaaa-3c84-4c46-8634-19541e735c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:05.275365 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 896/1793] 10.222.0.18 () {58 vars in 1015 bytes} [Sat Sep 10 14:14:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:05.344037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7301cb-1f8f-4223-af56-724403f4efca service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:05.370230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7301cb-1f8f-4223-af56-724403f4efca service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:05.416749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce1372c-33a2-4221-a5f5-997d1a7bd16b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:05.417387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1372c-33a2-4221-a5f5-997d1a7bd16b service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:05.417673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1372c-33a2-4221-a5f5-997d1a7bd16b service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:05.418069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce1372c-33a2-4221-a5f5-997d1a7bd16b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:05.488998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1372c-33a2-4221-a5f5-997d1a7bd16b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:05.499313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce1372c-33a2-4221-a5f5-997d1a7bd16b service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:05.543421 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 898/1794] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:05.599295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255aaaa-3c84-4c46-8634-19541e735c1a None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:05.601169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6255aaaa-3c84-4c46-8634-19541e735c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:05.640064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6255aaaa-3c84-4c46-8634-19541e735c1a None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:05.655189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:05.656392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:05.656956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:05.657441 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] PATH_INFO: `/v3/users/3c90d7bd370b4adb9cd5921846544431` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:05.659559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c90d7bd370b4adb9cd5921846544431)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:05.666560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:05.774194 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c90d7bd370b4adb9cd5921846544431'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:05.776301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c90d7bd370b4adb9cd5921846544431'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:05.792351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c90d7bd370b4adb9cd5921846544431'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:05.819575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c90d7bd370b4adb9cd5921846544431'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:05.821296 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c90d7bd370b4adb9cd5921846544431'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:05.853125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 897/1795] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:05 2022] DELETE /identity/v3/users/3c90d7bd370b4adb9cd5921846544431 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:05.858224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99245a00-3f24-45fb-ae93-818ebe329515 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:05.859079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99245a00-3f24-45fb-ae93-818ebe329515 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:05.859787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99245a00-3f24-45fb-ae93-818ebe329515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:05.894541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-99245a00-3f24-45fb-ae93-818ebe329515 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:05.896214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-99245a00-3f24-45fb-ae93-818ebe329515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:06.078026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 899/1796] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:06.115557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:06.117486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:06.360240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173dc8ca-b484-4575-820a-ab1df5255bae admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:06.367570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef870a4-f4a2-422e-8089-d874ad0d180b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:06.368398 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef870a4-f4a2-422e-8089-d874ad0d180b service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:06.369527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef870a4-f4a2-422e-8089-d874ad0d180b service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:06.369989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef870a4-f4a2-422e-8089-d874ad0d180b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:06.381783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99245a00-3f24-45fb-ae93-818ebe329515 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:06.383223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-99245a00-3f24-45fb-ae93-818ebe329515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:06.399823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef870a4-f4a2-422e-8089-d874ad0d180b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:06.403910 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99245a00-3f24-45fb-ae93-818ebe329515 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:06.410493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef870a4-f4a2-422e-8089-d874ad0d180b service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:06.444427 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:06.445311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:06.445600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:06.445882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] PATH_INFO: `/v3/projects/7ca8d0f1e9224b28b681d9c9c53a46ef` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:06.461144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca8d0f1e9224b28b681d9c9c53a46ef)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:06.474978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:06.476898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 898/1797] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:06.721266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca8d0f1e9224b28b681d9c9c53a46ef', 'request_id': 'req-f09acc12-97cc-4b4e-be38-592bee6ccb69'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:06.769614 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca8d0f1e9224b28b681d9c9c53a46ef', 'request_id': 'req-f09acc12-97cc-4b4e-be38-592bee6ccb69'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:06.801935 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca8d0f1e9224b28b681d9c9c53a46ef', 'request_id': 'req-f09acc12-97cc-4b4e-be38-592bee6ccb69'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:06.827990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] The token cache is being invalidate because project 7ca8d0f1e9224b28b681d9c9c53a46ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:06.828610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:06.848345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:06.859004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 900/1798] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:06 2022] DELETE /identity/v3/projects/7ca8d0f1e9224b28b681d9c9c53a46ef => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:06.862921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef870a4-f4a2-422e-8089-d874ad0d180b service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:06.890213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef870a4-f4a2-422e-8089-d874ad0d180b service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:06.924228 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:07.094405 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:07.095191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:07.095583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:07.095974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] PATH_INFO: `/v3/users/4fca43765baf4cdbb3c17c3dd7651db8` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:07.100252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fca43765baf4cdbb3c17c3dd7651db8)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:07.105680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:07.284873 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fca43765baf4cdbb3c17c3dd7651db8'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:07.285982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fca43765baf4cdbb3c17c3dd7651db8'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:07.311392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fca43765baf4cdbb3c17c3dd7651db8'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:07.335714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fca43765baf4cdbb3c17c3dd7651db8'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:07.336806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fca43765baf4cdbb3c17c3dd7651db8'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:07.363975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 899/1799] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:06 2022] DELETE /identity/v3/users/4fca43765baf4cdbb3c17c3dd7651db8 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:09.035737 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:09.037180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:09.058126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09acc12-97cc-4b4e-be38-592bee6ccb69 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:09.279607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:09.310800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:09.313954 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:09.323234 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:09.326863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] PATH_INFO: `/v3/users/f309030243944da6b933fd536a816591` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:09.328757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f309030243944da6b933fd536a816591)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:09.335208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:09.519048 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f309030243944da6b933fd536a816591'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:09.551788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f309030243944da6b933fd536a816591'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:09.554241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f309030243944da6b933fd536a816591'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:09.556942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f309030243944da6b933fd536a816591'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:09.618139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9d3a36d8-176e-4965-bb17-0643818a1f12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f309030243944da6b933fd536a816591'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:09.687539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:09.689080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:09.709160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f944b19-fa04-40f6-8edb-f706bc42a139 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:09.723294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 901/1800] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:09 2022] DELETE /identity/v3/users/f309030243944da6b933fd536a816591 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:09.729368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ff8e9-23b4-4250-9633-d7a0e7ed650c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:09.732583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ff8e9-23b4-4250-9633-d7a0e7ed650c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:09.733063 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03ff8e9-23b4-4250-9633-d7a0e7ed650c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:09.805293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d03ff8e9-23b4-4250-9633-d7a0e7ed650c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:09.806558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d03ff8e9-23b4-4250-9633-d7a0e7ed650c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:09.838404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 902/1801] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:09.845431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:09.848759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:09.849049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:09.849323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] PATH_INFO: `/v3/projects/fed001640cb44b989c8e4a486bc795ec` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:09.853234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fed001640cb44b989c8e4a486bc795ec)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:09.875688 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:09.897412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ff8e9-23b4-4250-9633-d7a0e7ed650c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:09.899296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d03ff8e9-23b4-4250-9633-d7a0e7ed650c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:10.000142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:10.040146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fed001640cb44b989c8e4a486bc795ec', 'request_id': 'req-52ddf107-12e9-4fa9-adf5-bb806f96d493'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:10.063423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fed001640cb44b989c8e4a486bc795ec', 'request_id': 'req-52ddf107-12e9-4fa9-adf5-bb806f96d493'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:10.074809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03ff8e9-23b4-4250-9633-d7a0e7ed650c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:10.083109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fed001640cb44b989c8e4a486bc795ec', 'request_id': 'req-52ddf107-12e9-4fa9-adf5-bb806f96d493'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:10.116099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] The token cache is being invalidate because project fed001640cb44b989c8e4a486bc795ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:10.116392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:10.120033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:10.129970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 900/1802] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:09 2022] DELETE /identity/v3/projects/fed001640cb44b989c8e4a486bc795ec => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:10.132604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:10.138826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:10.178840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec0b09-dd0a-4808-9e91-32aec0006da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:10.179506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec0b09-dd0a-4808-9e91-32aec0006da0 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:10.179792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec0b09-dd0a-4808-9e91-32aec0006da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:10.180065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec0b09-dd0a-4808-9e91-32aec0006da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:10.181938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a3ec0b09-dd0a-4808-9e91-32aec0006da0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:10.227501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:10.273003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec0b09-dd0a-4808-9e91-32aec0006da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:10.275004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec0b09-dd0a-4808-9e91-32aec0006da0 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:10.296766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 903/1803] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:10.391882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493a5d-0dae-4e19-aa58-da7697475dc6 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:10.392501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493a5d-0dae-4e19-aa58-da7697475dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:10.393100 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4493a5d-0dae-4e19-aa58-da7697475dc6 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:10.396991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 904/1804] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:14:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:14:10.404741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddf107-12e9-4fa9-adf5-bb806f96d493 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:10.412271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:10.413282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:10.413893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:10.414350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] PATH_INFO: `/v3/users/5cedd0cc77fd4eb7831cd1173e65f77e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:10.416575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cedd0cc77fd4eb7831cd1173e65f77e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:10.419564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:10.525243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cedd0cc77fd4eb7831cd1173e65f77e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:10.526170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cedd0cc77fd4eb7831cd1173e65f77e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:10.549269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cedd0cc77fd4eb7831cd1173e65f77e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:10.576356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cedd0cc77fd4eb7831cd1173e65f77e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:10.581194 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cedd0cc77fd4eb7831cd1173e65f77e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:10.663021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 901/1805] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:10 2022] DELETE /identity/v3/users/5cedd0cc77fd4eb7831cd1173e65f77e => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:10.677826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00dbfe-708d-446c-8bde-13715f7eba65 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:10.678118 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00dbfe-708d-446c-8bde-13715f7eba65 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:10.678408 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00dbfe-708d-446c-8bde-13715f7eba65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:10.736410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ab00dbfe-708d-446c-8bde-13715f7eba65 None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:10.755206 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ab00dbfe-708d-446c-8bde-13715f7eba65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:10.957880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 905/1806] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:14:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:11.258523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:11.260884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:11.386918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:11.441318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00dbfe-708d-446c-8bde-13715f7eba65 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:11.445047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ab00dbfe-708d-446c-8bde-13715f7eba65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:11.467743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5369d-7745-4afe-8a21-ca7e5228bf85 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:11.473691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:11.474602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:11.475045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:11.475564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] PATH_INFO: `/v3/projects/4d0c66d54f974e528e988a4f588725ab` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:11.477011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0c66d54f974e528e988a4f588725ab)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:11.477549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00dbfe-708d-446c-8bde-13715f7eba65 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:11.479651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:11.494327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc3c3e-fb8b-42ce-8062-2168170189ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:11.495117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc3c3e-fb8b-42ce-8062-2168170189ca service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:11.495538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc3c3e-fb8b-42ce-8062-2168170189ca service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:11.495988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc3c3e-fb8b-42ce-8062-2168170189ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:11.520323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc3c3e-fb8b-42ce-8062-2168170189ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:11.523601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc3c3e-fb8b-42ce-8062-2168170189ca service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:11.570222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 906/1807] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:11.575415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0c66d54f974e528e988a4f588725ab', 'request_id': 'req-e3964c90-75f4-49b6-afdc-629a803ee941'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:11.587393 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0c66d54f974e528e988a4f588725ab', 'request_id': 'req-e3964c90-75f4-49b6-afdc-629a803ee941'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:11.618627 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0c66d54f974e528e988a4f588725ab', 'request_id': 'req-e3964c90-75f4-49b6-afdc-629a803ee941'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:11.673922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] The token cache is being invalidate because project 4d0c66d54f974e528e988a4f588725ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:11.674355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:11.676274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3964c90-75f4-49b6-afdc-629a803ee941 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:11.685663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 902/1808] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:11 2022] DELETE /identity/v3/projects/4d0c66d54f974e528e988a4f588725ab => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:11.690413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5dc42-a604-40b4-86ee-aabce4c91dfd None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:11.690955 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5dc42-a604-40b4-86ee-aabce4c91dfd None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:11.691435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa5dc42-a604-40b4-86ee-aabce4c91dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:11.721883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2fa5dc42-a604-40b4-86ee-aabce4c91dfd None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:11.723701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2fa5dc42-a604-40b4-86ee-aabce4c91dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:11.936616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 907/1809] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:11.938846 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6ec53-0660-4081-947f-eda72c28d85f None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:11.939138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6ec53-0660-4081-947f-eda72c28d85f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:11.939417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d6ec53-0660-4081-947f-eda72c28d85f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:11.988554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a6d6ec53-0660-4081-947f-eda72c28d85f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:11.993247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a6d6ec53-0660-4081-947f-eda72c28d85f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:12.019202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 903/1810] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:12.021247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5dc42-a604-40b4-86ee-aabce4c91dfd None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:12.033086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa5dc42-a604-40b4-86ee-aabce4c91dfd None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:12.071032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df331c80-52a5-43c4-98df-cc739f743e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:12.073098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df331c80-52a5-43c4-98df-cc739f743e8a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:12.073381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df331c80-52a5-43c4-98df-cc739f743e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:12.073661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df331c80-52a5-43c4-98df-cc739f743e8a admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:12.074377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df331c80-52a5-43c4-98df-cc739f743e8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:12.074632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df331c80-52a5-43c4-98df-cc739f743e8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:12.076899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df331c80-52a5-43c4-98df-cc739f743e8a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:12.135618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 908/1811] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:12.136858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6ec53-0660-4081-947f-eda72c28d85f None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:12.149308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d6ec53-0660-4081-947f-eda72c28d85f None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:12.155993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daaf08-8724-47ae-8d3c-eed7b88743e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:12.156624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daaf08-8724-47ae-8d3c-eed7b88743e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:12.156903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daaf08-8724-47ae-8d3c-eed7b88743e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:12.157172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6daaf08-8724-47ae-8d3c-eed7b88743e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:12.157817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daaf08-8724-47ae-8d3c-eed7b88743e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:12.159860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6daaf08-8724-47ae-8d3c-eed7b88743e7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:12.221878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 904/1812] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:12 2022] POST /identity/v3/projects => generated 383 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:12.226146 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df331c80-52a5-43c4-98df-cc739f743e8a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:12.242655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df331c80-52a5-43c4-98df-cc739f743e8a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:12.338799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:12.374434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffbe1f-85a2-4705-80fa-595a46c0a4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:12.375428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffbe1f-85a2-4705-80fa-595a46c0a4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:12.375953 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffbe1f-85a2-4705-80fa-595a46c0a4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:12.377193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ffbe1f-85a2-4705-80fa-595a46c0a4b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:12.378088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffbe1f-85a2-4705-80fa-595a46c0a4b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:12.380587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ffbe1f-85a2-4705-80fa-595a46c0a4b5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:12.414096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 909/1813] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:12 2022] POST /identity/v3/users => generated 292 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:12.417728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daaf08-8724-47ae-8d3c-eed7b88743e7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:12.437515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6daaf08-8724-47ae-8d3c-eed7b88743e7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:12.444081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f63475-6446-4c4e-b50a-e235e690693c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:12.444707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f63475-6446-4c4e-b50a-e235e690693c admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:12.444984 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f63475-6446-4c4e-b50a-e235e690693c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:12.445254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f63475-6446-4c4e-b50a-e235e690693c admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:12.446278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f63475-6446-4c4e-b50a-e235e690693c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:12.447906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f63475-6446-4c4e-b50a-e235e690693c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:12.460676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 905/1814] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:12.465282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffbe1f-85a2-4705-80fa-595a46c0a4b5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:12.483506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ffbe1f-85a2-4705-80fa-595a46c0a4b5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:12.497281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53ec0e-f995-4586-859b-41a1f7552b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:12.498170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53ec0e-f995-4586-859b-41a1f7552b69 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:12.498634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53ec0e-f995-4586-859b-41a1f7552b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:12.498985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53ec0e-f995-4586-859b-41a1f7552b69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:12.500000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53ec0e-f995-4586-859b-41a1f7552b69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:12.501909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea53ec0e-f995-4586-859b-41a1f7552b69 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:12.531821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 910/1815] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:12 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:12.535234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f63475-6446-4c4e-b50a-e235e690693c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:12.552374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f63475-6446-4c4e-b50a-e235e690693c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:12.565548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8ff57-f5a1-48fd-9594-452607daf7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:12.566377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8ff57-f5a1-48fd-9594-452607daf7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:12.578485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8ff57-f5a1-48fd-9594-452607daf7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:12.579844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c8ff57-f5a1-48fd-9594-452607daf7fb admin admin] PATH_INFO: `/v3/projects/12b62aa5ee084451a3be1f771a678b69/users/7502adfdb9d0408392fc8b59448636ec/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:12.637426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8ff57-f5a1-48fd-9594-452607daf7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b62aa5ee084451a3be1f771a678b69, user_id=7502adfdb9d0408392fc8b59448636ec, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:12.649834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c8ff57-f5a1-48fd-9594-452607daf7fb admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:12.695417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 906/1816] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:12 2022] PUT /identity/v3/projects/12b62aa5ee084451a3be1f771a678b69/users/7502adfdb9d0408392fc8b59448636ec/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:12.706216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fbea5-3aec-446a-82b5-9821836cebdf None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:12.706745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fbea5-3aec-446a-82b5-9821836cebdf None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:12.707227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24fbea5-3aec-446a-82b5-9821836cebdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:12.783674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c24fbea5-3aec-446a-82b5-9821836cebdf None None] MFA Rules not processed for user `7502adfdb9d0408392fc8b59448636ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:12.785565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c24fbea5-3aec-446a-82b5-9821836cebdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:12.885711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8ff57-f5a1-48fd-9594-452607daf7fb admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:12.888381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-68c8ff57-f5a1-48fd-9594-452607daf7fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:12.922859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 911/1817] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:12.927288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fbea5-3aec-446a-82b5-9821836cebdf None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:13.092247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24fbea5-3aec-446a-82b5-9821836cebdf None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:13.098496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c8ff57-f5a1-48fd-9594-452607daf7fb admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:13.104894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e475e-1be7-428a-b443-8d3e275a0def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:13.105624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e475e-1be7-428a-b443-8d3e275a0def admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:13.105968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e475e-1be7-428a-b443-8d3e275a0def admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:13.106522 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e475e-1be7-428a-b443-8d3e275a0def admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:13.107279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e475e-1be7-428a-b443-8d3e275a0def admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:13.109360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e475e-1be7-428a-b443-8d3e275a0def admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:13.111839 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:13.112502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:13.115897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:13.115897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] PATH_INFO: `/v3/projects/9a6373074cbe471ab79e026e6ac40b84` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:13.118731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a6373074cbe471ab79e026e6ac40b84)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:13.125407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:13.154339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:13.168963 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 912/1818] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:12 2022] POST /identity/v3/projects => generated 381 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:13.170916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e475e-1be7-428a-b443-8d3e275a0def admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:13.240316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a6373074cbe471ab79e026e6ac40b84', 'request_id': 'req-ba48dfbf-0149-4dff-b452-553cdededfa3'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:13.254745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:13.255125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a6373074cbe471ab79e026e6ac40b84', 'request_id': 'req-ba48dfbf-0149-4dff-b452-553cdededfa3'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:13.266496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a6373074cbe471ab79e026e6ac40b84', 'request_id': 'req-ba48dfbf-0149-4dff-b452-553cdededfa3'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:13.296729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] The token cache is being invalidate because project 9a6373074cbe471ab79e026e6ac40b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:13.297029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:13.299397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ba48dfbf-0149-4dff-b452-553cdededfa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:13.301350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 907/1819] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:12 2022] DELETE /identity/v3/projects/9a6373074cbe471ab79e026e6ac40b84 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:13.306078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d401589-021b-489b-88bb-d85eafba8b53 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:13.306362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d401589-021b-489b-88bb-d85eafba8b53 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:13.306638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d401589-021b-489b-88bb-d85eafba8b53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:13.323118 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e475e-1be7-428a-b443-8d3e275a0def admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:13.352601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-5d401589-021b-489b-88bb-d85eafba8b53 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:13.353866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5d401589-021b-489b-88bb-d85eafba8b53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:13.486566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e54af8-55d2-4054-b14e-b762ced46e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:13.487367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e54af8-55d2-4054-b14e-b762ced46e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:13.487503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e54af8-55d2-4054-b14e-b762ced46e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:13.487724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e54af8-55d2-4054-b14e-b762ced46e34 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:13.488363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e54af8-55d2-4054-b14e-b762ced46e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:13.490322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e54af8-55d2-4054-b14e-b762ced46e34 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:13.502642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 908/1820] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:13.504459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca032de-c635-482c-9e87-d828c7d3d287 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:13.504730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca032de-c635-482c-9e87-d828c7d3d287 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:13.505056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca032de-c635-482c-9e87-d828c7d3d287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:13.541752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 913/1821] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:13 2022] POST /identity/v3/users => generated 297 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:13.547661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e54af8-55d2-4054-b14e-b762ced46e34 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:13.550850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f3e54af8-55d2-4054-b14e-b762ced46e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:13.553226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-dca032de-c635-482c-9e87-d828c7d3d287 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:13.554462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-dca032de-c635-482c-9e87-d828c7d3d287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:13.578474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 909/1822] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:13.578911 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e54af8-55d2-4054-b14e-b762ced46e34 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:13.579426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca032de-c635-482c-9e87-d828c7d3d287 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:13.592615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b032f27-e630-4b84-899d-ef932f6c0dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:13.593253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b032f27-e630-4b84-899d-ef932f6c0dac admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:13.593530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b032f27-e630-4b84-899d-ef932f6c0dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:13.593801 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b032f27-e630-4b84-899d-ef932f6c0dac admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:13.596745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b032f27-e630-4b84-899d-ef932f6c0dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:13.598447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b032f27-e630-4b84-899d-ef932f6c0dac admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:13.599190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca032de-c635-482c-9e87-d828c7d3d287 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:13.613371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4e4a4-7622-4979-8fb4-8ee06b55822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:13.614005 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4e4a4-7622-4979-8fb4-8ee06b55822f admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:13.614289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4e4a4-7622-4979-8fb4-8ee06b55822f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:13.614563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf4e4a4-7622-4979-8fb4-8ee06b55822f admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:13.615227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4e4a4-7622-4979-8fb4-8ee06b55822f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:13.615568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4e4a4-7622-4979-8fb4-8ee06b55822f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:13.617390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf4e4a4-7622-4979-8fb4-8ee06b55822f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:13.626226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 914/1823] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:13.632076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b032f27-e630-4b84-899d-ef932f6c0dac admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:13.650710 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 910/1824] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:13.651696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4e4a4-7622-4979-8fb4-8ee06b55822f admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:13.653627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b032f27-e630-4b84-899d-ef932f6c0dac admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:13.666643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b846ef-0d40-4d57-a368-6b60e265b05c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:13.668873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b846ef-0d40-4d57-a368-6b60e265b05c admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:13.669354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b846ef-0d40-4d57-a368-6b60e265b05c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:13.669834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b846ef-0d40-4d57-a368-6b60e265b05c admin admin] PATH_INFO: `/v3/projects/b2c1d30a18e1471999b60c14b685e536/users/0ca2f4c31ff44943982ad35dacc418bf/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:13.685266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf4e4a4-7622-4979-8fb4-8ee06b55822f admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:13.702563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b8c95-14b4-4c76-a7ba-7f4cad987da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:13.703338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b8c95-14b4-4c76-a7ba-7f4cad987da4 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:13.703721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b8c95-14b4-4c76-a7ba-7f4cad987da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:13.704003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b8c95-14b4-4c76-a7ba-7f4cad987da4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:13.704669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b8c95-14b4-4c76-a7ba-7f4cad987da4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:13.706635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395b8c95-14b4-4c76-a7ba-7f4cad987da4 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:13.720188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b846ef-0d40-4d57-a368-6b60e265b05c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c1d30a18e1471999b60c14b685e536, user_id=0ca2f4c31ff44943982ad35dacc418bf, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:13.728374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b846ef-0d40-4d57-a368-6b60e265b05c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:13.760542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 911/1825] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:13 2022] POST /identity/v3/projects => generated 393 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:13.760542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b8c95-14b4-4c76-a7ba-7f4cad987da4 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:13.766975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 915/1826] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:13 2022] PUT /identity/v3/projects/b2c1d30a18e1471999b60c14b685e536/users/0ca2f4c31ff44943982ad35dacc418bf/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:13.769146 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b846ef-0d40-4d57-a368-6b60e265b05c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:13.833628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:13.839596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395b8c95-14b4-4c76-a7ba-7f4cad987da4 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:13.959703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b846ef-0d40-4d57-a368-6b60e265b05c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:13.972910 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1f3ac-71cd-4fba-8a71-5414010e3155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:13.975083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1f3ac-71cd-4fba-8a71-5414010e3155 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:13.976630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1f3ac-71cd-4fba-8a71-5414010e3155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:13.977240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1f3ac-71cd-4fba-8a71-5414010e3155 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:13.978485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1f3ac-71cd-4fba-8a71-5414010e3155 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:13.982357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1f3ac-71cd-4fba-8a71-5414010e3155 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:14.002737 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 916/1827] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:14.007031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1f3ac-71cd-4fba-8a71-5414010e3155 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:14.031673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1f3ac-71cd-4fba-8a71-5414010e3155 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:14.051847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb2f86-c058-43fc-bc73-6c8b1fd09b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:14.053214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb2f86-c058-43fc-bc73-6c8b1fd09b83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:14.053787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb2f86-c058-43fc-bc73-6c8b1fd09b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:14.066232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb2f86-c058-43fc-bc73-6c8b1fd09b83 admin admin] PATH_INFO: `/v3/projects/b2c1d30a18e1471999b60c14b685e536/users/0ca2f4c31ff44943982ad35dacc418bf/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:14.070182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb2f86-c058-43fc-bc73-6c8b1fd09b83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2c1d30a18e1471999b60c14b685e536, user_id=0ca2f4c31ff44943982ad35dacc418bf, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:14.072824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bb706-9f54-4ff2-bcd4-4e791d26d865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:14.074770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb2f86-c058-43fc-bc73-6c8b1fd09b83 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:14.081817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb706-9f54-4ff2-bcd4-4e791d26d865 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:14.082150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb706-9f54-4ff2-bcd4-4e791d26d865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:14.082427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bb706-9f54-4ff2-bcd4-4e791d26d865 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:14.083382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bb706-9f54-4ff2-bcd4-4e791d26d865 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:14.089677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bb706-9f54-4ff2-bcd4-4e791d26d865 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:14.092862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 917/1828] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:14 2022] PUT /identity/v3/projects/b2c1d30a18e1471999b60c14b685e536/users/0ca2f4c31ff44943982ad35dacc418bf/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:14.112443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5a9e5-444f-4bf9-9080-830d94265d3a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:14.112818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5a9e5-444f-4bf9-9080-830d94265d3a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:14.113257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5a9e5-444f-4bf9-9080-830d94265d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:14.149635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 912/1829] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:13 2022] POST /identity/v3/users => generated 297 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:14.154447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bb706-9f54-4ff2-bcd4-4e791d26d865 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:14.164166 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a1f5a9e5-444f-4bf9-9080-830d94265d3a None None] MFA Rules not processed for user `0ca2f4c31ff44943982ad35dacc418bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:14.165513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a1f5a9e5-444f-4bf9-9080-830d94265d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:14.171617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:14.194247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bb706-9f54-4ff2-bcd4-4e791d26d865 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:14.365886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 918/1830] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:14 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:14.372705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e51c2-68a5-4ae8-855f-fad590863d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:14.373343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e51c2-68a5-4ae8-855f-fad590863d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:14.373620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e51c2-68a5-4ae8-855f-fad590863d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:14.373953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e51c2-68a5-4ae8-855f-fad590863d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:14.374708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e51c2-68a5-4ae8-855f-fad590863d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:14.376635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55e51c2-68a5-4ae8-855f-fad590863d05 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:14.377036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5a9e5-444f-4bf9-9080-830d94265d3a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:14.378785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a1f5a9e5-444f-4bf9-9080-830d94265d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:14.402991 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 913/1831] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:14.408431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e51c2-68a5-4ae8-855f-fad590863d05 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:14.433726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e51c2-68a5-4ae8-855f-fad590863d05 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:14.448641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe4da8-6243-4268-8b41-189bb4667f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:14.449268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe4da8-6243-4268-8b41-189bb4667f44 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:14.450492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe4da8-6243-4268-8b41-189bb4667f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:14.450492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe4da8-6243-4268-8b41-189bb4667f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:14.451063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe4da8-6243-4268-8b41-189bb4667f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:14.452820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe4da8-6243-4268-8b41-189bb4667f44 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:14.484377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 914/1832] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:14.489098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe4da8-6243-4268-8b41-189bb4667f44 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:14.511824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe4da8-6243-4268-8b41-189bb4667f44 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:14.525341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e304958-2f9d-4129-a188-13aa6e363aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:14.526009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e304958-2f9d-4129-a188-13aa6e363aee admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:14.529968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e304958-2f9d-4129-a188-13aa6e363aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:14.530764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e304958-2f9d-4129-a188-13aa6e363aee admin admin] PATH_INFO: `/v3/projects/f7a1837eb73444529dfe072a69d7d670/users/f79e3ae5e01d4e35b46dffdb48798e24/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:14.540733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5a9e5-444f-4bf9-9080-830d94265d3a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:14.544856 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76634f-706d-4e4e-9aea-4dbf2797e0b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:14.546843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76634f-706d-4e4e-9aea-4dbf2797e0b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:14.547287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76634f-706d-4e4e-9aea-4dbf2797e0b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:14.547790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c76634f-706d-4e4e-9aea-4dbf2797e0b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:14.573537 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e304958-2f9d-4129-a188-13aa6e363aee admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a1837eb73444529dfe072a69d7d670, user_id=f79e3ae5e01d4e35b46dffdb48798e24, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:14.576743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76634f-706d-4e4e-9aea-4dbf2797e0b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:14.578243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e304958-2f9d-4129-a188-13aa6e363aee admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:14.579377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c76634f-706d-4e4e-9aea-4dbf2797e0b3 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:14.616202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 915/1833] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:14 2022] PUT /identity/v3/projects/f7a1837eb73444529dfe072a69d7d670/users/f79e3ae5e01d4e35b46dffdb48798e24/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:14.627708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b1992-6172-437c-8de9-72ac9a13c614 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:14.628301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b1992-6172-437c-8de9-72ac9a13c614 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:14.635904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b1992-6172-437c-8de9-72ac9a13c614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:14.646736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 919/1834] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:14 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:14.726034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-021b1992-6172-437c-8de9-72ac9a13c614 None None] MFA Rules not processed for user `f79e3ae5e01d4e35b46dffdb48798e24`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:14.727271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-021b1992-6172-437c-8de9-72ac9a13c614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:14.818983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76634f-706d-4e4e-9aea-4dbf2797e0b3 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:14.822870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:14.837815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c76634f-706d-4e4e-9aea-4dbf2797e0b3 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:14.860394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 916/1835] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:14.873878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b1992-6172-437c-8de9-72ac9a13c614 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:14.973895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b452a-2c70-4e9c-b609-95aa6d4270e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:14.976274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b452a-2c70-4e9c-b609-95aa6d4270e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:14.977123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b452a-2c70-4e9c-b609-95aa6d4270e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:14.977772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b452a-2c70-4e9c-b609-95aa6d4270e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:14.989518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9a2b452a-2c70-4e9c-b609-95aa6d4270e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:15.109583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b1992-6172-437c-8de9-72ac9a13c614 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:15.142183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716089d-cbaa-41c1-a516-1d98f871195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:15.143945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716089d-cbaa-41c1-a516-1d98f871195d admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:15.147002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716089d-cbaa-41c1-a516-1d98f871195d admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:15.148761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716089d-cbaa-41c1-a516-1d98f871195d admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:15.149841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716089d-cbaa-41c1-a516-1d98f871195d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:15.166197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716089d-cbaa-41c1-a516-1d98f871195d admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:15.233349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b452a-2c70-4e9c-b609-95aa6d4270e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:15.237950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b452a-2c70-4e9c-b609-95aa6d4270e0 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:15.249151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:15.260234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 917/1836] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:14 2022] POST /identity/v3/projects => generated 391 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:15.263527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716089d-cbaa-41c1-a516-1d98f871195d admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:15.265250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0716089d-cbaa-41c1-a516-1d98f871195d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:15.268161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 920/1837] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:15.285738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b452a-2c70-4e9c-b609-95aa6d4270e0 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:15.323860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b452a-2c70-4e9c-b609-95aa6d4270e0 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:15.454439 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b915e4-a97c-4594-a185-3bdac2e3643c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:15.456047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b915e4-a97c-4594-a185-3bdac2e3643c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:15.456481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b915e4-a97c-4594-a185-3bdac2e3643c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:15.456897 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b915e4-a97c-4594-a185-3bdac2e3643c admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:15.457683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b915e4-a97c-4594-a185-3bdac2e3643c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:15.458840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716089d-cbaa-41c1-a516-1d98f871195d admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:15.460851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b915e4-a97c-4594-a185-3bdac2e3643c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:15.470323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:15.471626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:15.471916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:15.472592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] PATH_INFO: `/v3/projects/105a7d6b44974874aeda9e0fe2b5f112` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:15.476044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=105a7d6b44974874aeda9e0fe2b5f112)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:15.483118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:15.493351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 921/1838] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:15 2022] POST /identity/v3/users => generated 302 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:15.498296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b915e4-a97c-4594-a185-3bdac2e3643c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:15.514043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b915e4-a97c-4594-a185-3bdac2e3643c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:15.522852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5a0d7-f88e-4c71-b336-b0c073e85f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:15.524348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a0d7-f88e-4c71-b336-b0c073e85f78 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:15.524633 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a0d7-f88e-4c71-b336-b0c073e85f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:15.525598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b5a0d7-f88e-4c71-b336-b0c073e85f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:15.526778 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5a0d7-f88e-4c71-b336-b0c073e85f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:15.571027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b5a0d7-f88e-4c71-b336-b0c073e85f78 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:15.579512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 922/1839] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:15.581058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5a0d7-f88e-4c71-b336-b0c073e85f78 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:15.599657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b5a0d7-f88e-4c71-b336-b0c073e85f78 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:15.615074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '105a7d6b44974874aeda9e0fe2b5f112', 'request_id': 'req-7a278952-3575-43fa-b795-eb4668492a80'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:15.641372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '105a7d6b44974874aeda9e0fe2b5f112', 'request_id': 'req-7a278952-3575-43fa-b795-eb4668492a80'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:15.656404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '105a7d6b44974874aeda9e0fe2b5f112', 'request_id': 'req-7a278952-3575-43fa-b795-eb4668492a80'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:15.674125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:15.682962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] The token cache is being invalidate because project 105a7d6b44974874aeda9e0fe2b5f112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:15.683643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:15.685096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:15.690886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 918/1840] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:15 2022] DELETE /identity/v3/projects/105a7d6b44974874aeda9e0fe2b5f112 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:15.692043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:15.693577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:15.754863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5c712-54a4-4dff-b2ed-6e7db1b04e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:15.756217 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5c712-54a4-4dff-b2ed-6e7db1b04e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:15.756540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5c712-54a4-4dff-b2ed-6e7db1b04e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:15.757372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5c712-54a4-4dff-b2ed-6e7db1b04e8d admin admin] PATH_INFO: `/v3/projects/33d2d40c9ef54d3394886dffae08c253/users/7bceb3abc4ca4e64b0e2964de1f01233/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:15.825952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a278952-3575-43fa-b795-eb4668492a80 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:15.830181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5c712-54a4-4dff-b2ed-6e7db1b04e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d2d40c9ef54d3394886dffae08c253, user_id=7bceb3abc4ca4e64b0e2964de1f01233, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:15.837044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee5c712-54a4-4dff-b2ed-6e7db1b04e8d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:15.837909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:15.838608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:15.839008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:15.839168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] PATH_INFO: `/v3/users/cfb18830a827477b9ad7353450191c15` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:15.840483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfb18830a827477b9ad7353450191c15)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:15.842942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:15.904117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 923/1841] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:15 2022] PUT /identity/v3/projects/33d2d40c9ef54d3394886dffae08c253/users/7bceb3abc4ca4e64b0e2964de1f01233/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:15.905361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5c712-54a4-4dff-b2ed-6e7db1b04e8d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:15.908033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8ee5c712-54a4-4dff-b2ed-6e7db1b04e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:15.928924 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfb18830a827477b9ad7353450191c15'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:15.931118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfb18830a827477b9ad7353450191c15'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:15.953677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfb18830a827477b9ad7353450191c15'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:15.976612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfb18830a827477b9ad7353450191c15'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:15.990214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfb18830a827477b9ad7353450191c15'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:16.020445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 919/1842] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:15 2022] DELETE /identity/v3/users/cfb18830a827477b9ad7353450191c15 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:16.029765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee5c712-54a4-4dff-b2ed-6e7db1b04e8d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:16.035301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:16.143061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc08837f-7d29-4d6c-bff4-6d2679dd415e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:16.144120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08837f-7d29-4d6c-bff4-6d2679dd415e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:16.144594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08837f-7d29-4d6c-bff4-6d2679dd415e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:16.145055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc08837f-7d29-4d6c-bff4-6d2679dd415e admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:16.146041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08837f-7d29-4d6c-bff4-6d2679dd415e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:16.148221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc08837f-7d29-4d6c-bff4-6d2679dd415e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:16.163371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 924/1843] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:16.167368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:16.171015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:16.200434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c0c5b6-11a8-4474-9373-d4dc9f6e4cfd admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:16.206038 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada0f058-8472-4516-b013-cfd500771333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:16.206827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0f058-8472-4516-b013-cfd500771333 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:16.207209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0f058-8472-4516-b013-cfd500771333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:16.207627 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0f058-8472-4516-b013-cfd500771333 admin admin] PATH_INFO: `/v3/projects/33d2d40c9ef54d3394886dffae08c253/users/7bceb3abc4ca4e64b0e2964de1f01233/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:16.210812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0f058-8472-4516-b013-cfd500771333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d2d40c9ef54d3394886dffae08c253, user_id=7bceb3abc4ca4e64b0e2964de1f01233, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:16.213931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada0f058-8472-4516-b013-cfd500771333 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:16.226794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 920/1844] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:16 2022] PUT /identity/v3/projects/33d2d40c9ef54d3394886dffae08c253/users/7bceb3abc4ca4e64b0e2964de1f01233/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:16.238119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9b530-8c3a-4160-9311-8bc87e9e2d4e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:16.238401 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9b530-8c3a-4160-9311-8bc87e9e2d4e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:16.238674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9b530-8c3a-4160-9311-8bc87e9e2d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:16.275583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e2a9b530-8c3a-4160-9311-8bc87e9e2d4e None None] MFA Rules not processed for user `7bceb3abc4ca4e64b0e2964de1f01233`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:16.278850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e2a9b530-8c3a-4160-9311-8bc87e9e2d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:16.351528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3615392-b924-4246-8234-51738ae22d12 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:16.352782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3615392-b924-4246-8234-51738ae22d12 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:16.353188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3615392-b924-4246-8234-51738ae22d12 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:16.354939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 921/1845] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:14:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:14:16.393171 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd21e42-d867-4bbb-8458-a5613bdf7934 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:16.396044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd21e42-d867-4bbb-8458-a5613bdf7934 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:16.397701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd21e42-d867-4bbb-8458-a5613bdf7934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:16.407944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 925/1846] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:16 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:16.428260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9b530-8c3a-4160-9311-8bc87e9e2d4e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:16.430820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e2a9b530-8c3a-4160-9311-8bc87e9e2d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:16.474774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-edd21e42-d867-4bbb-8458-a5613bdf7934 None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:16.476935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-edd21e42-d867-4bbb-8458-a5613bdf7934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:16.545183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9b530-8c3a-4160-9311-8bc87e9e2d4e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:16.555008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08287de4-c46d-46fa-9ef1-ef9eb3353bfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:16.557813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08287de4-c46d-46fa-9ef1-ef9eb3353bfe service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:16.558204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08287de4-c46d-46fa-9ef1-ef9eb3353bfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:16.561084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08287de4-c46d-46fa-9ef1-ef9eb3353bfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:16.577515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:16.610110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 922/1847] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:14:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:16.611101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08287de4-c46d-46fa-9ef1-ef9eb3353bfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:16.612737 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08287de4-c46d-46fa-9ef1-ef9eb3353bfe service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:16.630356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 926/1848] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:16 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:16.703955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd21e42-d867-4bbb-8458-a5613bdf7934 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:16.705429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-edd21e42-d867-4bbb-8458-a5613bdf7934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:16.732003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd21e42-d867-4bbb-8458-a5613bdf7934 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:16.739085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08287de4-c46d-46fa-9ef1-ef9eb3353bfe service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:16.743128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-08287de4-c46d-46fa-9ef1-ef9eb3353bfe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:16.750474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9c59c-910e-40d1-a601-c7e1b49f0d61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:16.753939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9c59c-910e-40d1-a601-c7e1b49f0d61 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:16.753939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9c59c-910e-40d1-a601-c7e1b49f0d61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:16.753939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9c59c-910e-40d1-a601-c7e1b49f0d61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:16.756648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0dc9c59c-910e-40d1-a601-c7e1b49f0d61 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:16.766374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08287de4-c46d-46fa-9ef1-ef9eb3353bfe service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:16.773889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24231b49-14fc-4ee5-ab88-7ca99ca16d48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:16.774885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24231b49-14fc-4ee5-ab88-7ca99ca16d48 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:16.775242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24231b49-14fc-4ee5-ab88-7ca99ca16d48 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:16.775432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24231b49-14fc-4ee5-ab88-7ca99ca16d48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:16.787785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24231b49-14fc-4ee5-ab88-7ca99ca16d48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:16.789728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24231b49-14fc-4ee5-ab88-7ca99ca16d48 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:16.815639 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 927/1849] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:16.839589 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9c59c-910e-40d1-a601-c7e1b49f0d61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:16.841263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9c59c-910e-40d1-a601-c7e1b49f0d61 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:16.852578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 923/1850] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:17.405585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24231b49-14fc-4ee5-ab88-7ca99ca16d48 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:17.407009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-24231b49-14fc-4ee5-ab88-7ca99ca16d48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:17.461952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:17.523568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24231b49-14fc-4ee5-ab88-7ca99ca16d48 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:17.528295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:17.529057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:17.529340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:17.529618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] PATH_INFO: `/v3/projects/f802221799194033be2644ccf5198f9b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:17.530751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f802221799194033be2644ccf5198f9b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:17.532744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:17.583149 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f802221799194033be2644ccf5198f9b', 'request_id': 'req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:17.589441 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f802221799194033be2644ccf5198f9b', 'request_id': 'req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:17.599995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f802221799194033be2644ccf5198f9b', 'request_id': 'req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:17.613604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] The token cache is being invalidate because project f802221799194033be2644ccf5198f9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:17.614270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:17.614557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3722fe2-053c-433f-9ab4-dbb9fc75dc47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:17.620021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 928/1851] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:17 2022] DELETE /identity/v3/projects/f802221799194033be2644ccf5198f9b => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:17.621728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30241285-0d98-48cf-9c10-8ad4ba9725c7 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:17.622166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30241285-0d98-48cf-9c10-8ad4ba9725c7 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:17.622479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30241285-0d98-48cf-9c10-8ad4ba9725c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:17.644636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-30241285-0d98-48cf-9c10-8ad4ba9725c7 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:17.645895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-30241285-0d98-48cf-9c10-8ad4ba9725c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:17.728548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 924/1852] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:17.737565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fb631-ed52-4bcb-96fb-e3beacaab21c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:17.737851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fb631-ed52-4bcb-96fb-e3beacaab21c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:17.738129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214fb631-ed52-4bcb-96fb-e3beacaab21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:17.768663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-214fb631-ed52-4bcb-96fb-e3beacaab21c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:17.770643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-214fb631-ed52-4bcb-96fb-e3beacaab21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:17.800942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 929/1853] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:17.814164 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30241285-0d98-48cf-9c10-8ad4ba9725c7 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:17.824098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30241285-0d98-48cf-9c10-8ad4ba9725c7 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:17.830001 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ff248-803d-4ae3-a90b-08f0a4c7b1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:17.830844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ff248-803d-4ae3-a90b-08f0a4c7b1b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:17.831286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ff248-803d-4ae3-a90b-08f0a4c7b1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:17.831776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560ff248-803d-4ae3-a90b-08f0a4c7b1b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:17.832656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ff248-803d-4ae3-a90b-08f0a4c7b1b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:17.833185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ff248-803d-4ae3-a90b-08f0a4c7b1b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:17.835107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560ff248-803d-4ae3-a90b-08f0a4c7b1b1 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:17.852414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 925/1854] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:17.853620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fb631-ed52-4bcb-96fb-e3beacaab21c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:17.868008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214fb631-ed52-4bcb-96fb-e3beacaab21c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:17.874263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edc428-e304-417a-afa9-52b6ee628339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:17.874456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edc428-e304-417a-afa9-52b6ee628339 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:17.874725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edc428-e304-417a-afa9-52b6ee628339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:17.874997 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0edc428-e304-417a-afa9-52b6ee628339 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:17.877849 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edc428-e304-417a-afa9-52b6ee628339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:17.877849 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0edc428-e304-417a-afa9-52b6ee628339 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:17.910450 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 930/1855] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:17 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:17.914599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ff248-803d-4ae3-a90b-08f0a4c7b1b1 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:17.928423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560ff248-803d-4ae3-a90b-08f0a4c7b1b1 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.002212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a449a8-17e9-484e-9173-fbb407ad5365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.002923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a449a8-17e9-484e-9173-fbb407ad5365 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.003275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a449a8-17e9-484e-9173-fbb407ad5365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.004023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a449a8-17e9-484e-9173-fbb407ad5365 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.004374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a449a8-17e9-484e-9173-fbb407ad5365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.006428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a449a8-17e9-484e-9173-fbb407ad5365 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.031652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 926/1856] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:17 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:18.035667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edc428-e304-417a-afa9-52b6ee628339 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.046611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0edc428-e304-417a-afa9-52b6ee628339 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.052178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75c94c-94a7-4e20-a3d1-8e84c1c56162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.052789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75c94c-94a7-4e20-a3d1-8e84c1c56162 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.053063 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75c94c-94a7-4e20-a3d1-8e84c1c56162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.053332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75c94c-94a7-4e20-a3d1-8e84c1c56162 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.054021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75c94c-94a7-4e20-a3d1-8e84c1c56162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.055614 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75c94c-94a7-4e20-a3d1-8e84c1c56162 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.067899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 931/1857] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:18.071780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a449a8-17e9-484e-9173-fbb407ad5365 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.088497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a449a8-17e9-484e-9173-fbb407ad5365 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.094856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486ef4f-eaf1-4d6f-8c31-73e2b6ca83d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.095512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ef4f-eaf1-4d6f-8c31-73e2b6ca83d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.096203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ef4f-eaf1-4d6f-8c31-73e2b6ca83d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.096482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486ef4f-eaf1-4d6f-8c31-73e2b6ca83d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.097379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486ef4f-eaf1-4d6f-8c31-73e2b6ca83d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.099331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d486ef4f-eaf1-4d6f-8c31-73e2b6ca83d2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.112453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 927/1858] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:18.117309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75c94c-94a7-4e20-a3d1-8e84c1c56162 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.129282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75c94c-94a7-4e20-a3d1-8e84c1c56162 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.134188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90915c-3afd-4898-b825-399695e062ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.134834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90915c-3afd-4898-b825-399695e062ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.135107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90915c-3afd-4898-b825-399695e062ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.135377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90915c-3afd-4898-b825-399695e062ee admin admin] PATH_INFO: `/v3/projects/6cb913ede30047cdb38d4c524885b376/users/b2da365f5d5346db95a21732fe68161f/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.179800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90915c-3afd-4898-b825-399695e062ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb913ede30047cdb38d4c524885b376, user_id=b2da365f5d5346db95a21732fe68161f, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.179800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90915c-3afd-4898-b825-399695e062ee admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.191429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 932/1859] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:18 2022] PUT /identity/v3/projects/6cb913ede30047cdb38d4c524885b376/users/b2da365f5d5346db95a21732fe68161f/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:18.202911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d375ce-e1c9-417f-a9ad-b52f7c1d5bc5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.203198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d375ce-e1c9-417f-a9ad-b52f7c1d5bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.203569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d375ce-e1c9-417f-a9ad-b52f7c1d5bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.211347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:18.235042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e3d375ce-e1c9-417f-a9ad-b52f7c1d5bc5 None None] MFA Rules not processed for user `b2da365f5d5346db95a21732fe68161f`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:18.237212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e3d375ce-e1c9-417f-a9ad-b52f7c1d5bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:18.316068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 928/1860] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:18.320931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90915c-3afd-4898-b825-399695e062ee admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.373634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:18.397280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90915c-3afd-4898-b825-399695e062ee admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.402808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896a6be-a691-4cf8-a551-a971e794ecde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.403172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896a6be-a691-4cf8-a551-a971e794ecde admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.403538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896a6be-a691-4cf8-a551-a971e794ecde admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.403728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896a6be-a691-4cf8-a551-a971e794ecde admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.404436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896a6be-a691-4cf8-a551-a971e794ecde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.406611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896a6be-a691-4cf8-a551-a971e794ecde admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.444384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 933/1861] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:18 2022] POST /identity/v3/projects => generated 379 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:18.448105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d375ce-e1c9-417f-a9ad-b52f7c1d5bc5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.465927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d375ce-e1c9-417f-a9ad-b52f7c1d5bc5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.484776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896a6be-a691-4cf8-a551-a971e794ecde admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.488491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7896a6be-a691-4cf8-a551-a971e794ecde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:18.553080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174de4a-2103-436c-9ca8-403750fe494a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.553827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174de4a-2103-436c-9ca8-403750fe494a admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.554242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174de4a-2103-436c-9ca8-403750fe494a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.554563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174de4a-2103-436c-9ca8-403750fe494a admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.555200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174de4a-2103-436c-9ca8-403750fe494a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.563628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9174de4a-2103-436c-9ca8-403750fe494a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.589099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896a6be-a691-4cf8-a551-a971e794ecde admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.590295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 929/1862] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:18 2022] POST /identity/v3/users => generated 290 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:18.594201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174de4a-2103-436c-9ca8-403750fe494a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.596959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.597596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.597878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.598154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] PATH_INFO: `/v3/users/7502adfdb9d0408392fc8b59448636ec` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.599293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] RBAC: Authorizing `identity:delete_user(user_id=7502adfdb9d0408392fc8b59448636ec)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.601309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.604867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174de4a-2103-436c-9ca8-403750fe494a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.610253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ac553-c707-4e09-ba1d-f5f8507608ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.610955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ac553-c707-4e09-ba1d-f5f8507608ef admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.611292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ac553-c707-4e09-ba1d-f5f8507608ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.611783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ac553-c707-4e09-ba1d-f5f8507608ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.612483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ac553-c707-4e09-ba1d-f5f8507608ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.614132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ac553-c707-4e09-ba1d-f5f8507608ef admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.625110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 930/1863] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:18.629928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ac553-c707-4e09-ba1d-f5f8507608ef admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.646074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ac553-c707-4e09-ba1d-f5f8507608ef admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.650715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc71e7e-6fb5-4a5f-b3a1-5b98236832c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.651340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc71e7e-6fb5-4a5f-b3a1-5b98236832c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.651770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc71e7e-6fb5-4a5f-b3a1-5b98236832c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.652050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc71e7e-6fb5-4a5f-b3a1-5b98236832c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.652738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc71e7e-6fb5-4a5f-b3a1-5b98236832c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.654637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc71e7e-6fb5-4a5f-b3a1-5b98236832c9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.662365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7502adfdb9d0408392fc8b59448636ec'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:18.667791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 931/1864] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:18.672589 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc71e7e-6fb5-4a5f-b3a1-5b98236832c9 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.676105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7502adfdb9d0408392fc8b59448636ec'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:18.676730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7502adfdb9d0408392fc8b59448636ec'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:18.678444 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7502adfdb9d0408392fc8b59448636ec'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:18.683226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc71e7e-6fb5-4a5f-b3a1-5b98236832c9 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.688560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a8ec5d2-314f-4025-8e91-4c578833febb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7502adfdb9d0408392fc8b59448636ec'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:18.689004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2ce49c-5c17-4a35-b9da-3833d4779df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.689451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ce49c-5c17-4a35-b9da-3833d4779df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.689763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ce49c-5c17-4a35-b9da-3833d4779df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.690053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2ce49c-5c17-4a35-b9da-3833d4779df3 admin admin] PATH_INFO: `/v3/projects/942d783d585c484985d0f43f36360d42/users/cc7124871914409e8d3aaefc0eb183fa/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.706819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ce49c-5c17-4a35-b9da-3833d4779df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=942d783d585c484985d0f43f36360d42, user_id=cc7124871914409e8d3aaefc0eb183fa, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.710978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 934/1865] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:18 2022] DELETE /identity/v3/users/7502adfdb9d0408392fc8b59448636ec => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:18.711126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2ce49c-5c17-4a35-b9da-3833d4779df3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.714920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f2de2-4e7f-402a-b3d4-9affbe64136e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.715210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f2de2-4e7f-402a-b3d4-9affbe64136e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.715713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f2de2-4e7f-402a-b3d4-9affbe64136e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.724802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 932/1866] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:18 2022] PUT /identity/v3/projects/942d783d585c484985d0f43f36360d42/users/cc7124871914409e8d3aaefc0eb183fa/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:18.737242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ded8f1-cb97-4272-a56a-2c95b27dfef1 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.737834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ded8f1-cb97-4272-a56a-2c95b27dfef1 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.737915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-499f2de2-4e7f-402a-b3d4-9affbe64136e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:18.737992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ded8f1-cb97-4272-a56a-2c95b27dfef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.738733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-499f2de2-4e7f-402a-b3d4-9affbe64136e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:18.772402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-b7ded8f1-cb97-4272-a56a-2c95b27dfef1 None None] MFA Rules not processed for user `cc7124871914409e8d3aaefc0eb183fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:18.773768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b7ded8f1-cb97-4272-a56a-2c95b27dfef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:18.798981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:18.802405 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:18.836601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 935/1867] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:18.850033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f2de2-4e7f-402a-b3d4-9affbe64136e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.852240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-499f2de2-4e7f-402a-b3d4-9affbe64136e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:18.868735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 933/1868] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:18.874088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ded8f1-cb97-4272-a56a-2c95b27dfef1 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.875496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b7ded8f1-cb97-4272-a56a-2c95b27dfef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:18.889758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ded8f1-cb97-4272-a56a-2c95b27dfef1 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:18.894746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06d416-b0b0-4299-b9ea-dde0709d227f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:18.895370 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06d416-b0b0-4299-b9ea-dde0709d227f admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:18.895681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06d416-b0b0-4299-b9ea-dde0709d227f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:18.895960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06d416-b0b0-4299-b9ea-dde0709d227f admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:18.896607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06d416-b0b0-4299-b9ea-dde0709d227f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:18.898527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06d416-b0b0-4299-b9ea-dde0709d227f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:18.942431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 934/1869] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:18 2022] POST /identity/v3/projects => generated 377 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:18.943483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06d416-b0b0-4299-b9ea-dde0709d227f admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:18.945934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f2de2-4e7f-402a-b3d4-9affbe64136e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:19.024718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2c4c9-abfa-45a2-8434-4f12a94532e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:19.025445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c4c9-abfa-45a2-8434-4f12a94532e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.025975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c4c9-abfa-45a2-8434-4f12a94532e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.027604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a2c4c9-abfa-45a2-8434-4f12a94532e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.049358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06d416-b0b0-4299-b9ea-dde0709d227f admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:19.054377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528c40c-95d5-45c3-86fa-c0dbb6da9ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:19.055199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528c40c-95d5-45c3-86fa-c0dbb6da9ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.055509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528c40c-95d5-45c3-86fa-c0dbb6da9ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.055785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528c40c-95d5-45c3-86fa-c0dbb6da9ca7 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.056416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528c40c-95d5-45c3-86fa-c0dbb6da9ca7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:19.061220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528c40c-95d5-45c3-86fa-c0dbb6da9ca7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:19.092262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 935/1870] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:18 2022] POST /identity/v3/users => generated 295 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:19.093587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528c40c-95d5-45c3-86fa-c0dbb6da9ca7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:19.103998 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528c40c-95d5-45c3-86fa-c0dbb6da9ca7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:19.109378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d80c7-8123-4ad6-aa5f-8ebfed75513e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:19.110149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d80c7-8123-4ad6-aa5f-8ebfed75513e admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.110454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d80c7-8123-4ad6-aa5f-8ebfed75513e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.110800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d80c7-8123-4ad6-aa5f-8ebfed75513e admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.111554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d80c7-8123-4ad6-aa5f-8ebfed75513e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:19.113398 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d80c7-8123-4ad6-aa5f-8ebfed75513e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:19.114477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2c4c9-abfa-45a2-8434-4f12a94532e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:19.116239 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a2c4c9-abfa-45a2-8434-4f12a94532e8 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:19.126315 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 936/1871] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:19.130634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d80c7-8123-4ad6-aa5f-8ebfed75513e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:19.141645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 936/1872] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:19.142932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d80c7-8123-4ad6-aa5f-8ebfed75513e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:19.153142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87342de2-57e2-47ca-864e-cb5c484c64ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:19.154633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87342de2-57e2-47ca-864e-cb5c484c64ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.155246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87342de2-57e2-47ca-864e-cb5c484c64ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.156008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87342de2-57e2-47ca-864e-cb5c484c64ac admin admin] PATH_INFO: `/v3/projects/37593cd9f66b4848b0021e556c6a75cf/users/29b26036a0834f30973fc4501390d3e3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.182662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87342de2-57e2-47ca-864e-cb5c484c64ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=37593cd9f66b4848b0021e556c6a75cf, user_id=29b26036a0834f30973fc4501390d3e3, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:19.187939 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87342de2-57e2-47ca-864e-cb5c484c64ac admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:19.211193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 937/1873] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:19 2022] PUT /identity/v3/projects/37593cd9f66b4848b0021e556c6a75cf/users/29b26036a0834f30973fc4501390d3e3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:19.215903 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2c4c9-abfa-45a2-8434-4f12a94532e8 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:19.234150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a2c4c9-abfa-45a2-8434-4f12a94532e8 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:19.315383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:19.324477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fdb801-ad17-471d-95e8-8cf20650f77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:19.326641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb801-ad17-471d-95e8-8cf20650f77b admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.326923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb801-ad17-471d-95e8-8cf20650f77b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.327196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fdb801-ad17-471d-95e8-8cf20650f77b admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.327923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdb801-ad17-471d-95e8-8cf20650f77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:19.329548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fdb801-ad17-471d-95e8-8cf20650f77b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:19.343929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 937/1874] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:19.345176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87342de2-57e2-47ca-864e-cb5c484c64ac admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:19.355937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87342de2-57e2-47ca-864e-cb5c484c64ac admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:19.362205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59016b-4490-4226-8ba8-1c4a31634c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:19.363066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59016b-4490-4226-8ba8-1c4a31634c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.363617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59016b-4490-4226-8ba8-1c4a31634c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.364044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59016b-4490-4226-8ba8-1c4a31634c53 admin admin] PATH_INFO: `/v3/projects/37593cd9f66b4848b0021e556c6a75cf/users/29b26036a0834f30973fc4501390d3e3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.366104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59016b-4490-4226-8ba8-1c4a31634c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37593cd9f66b4848b0021e556c6a75cf, user_id=29b26036a0834f30973fc4501390d3e3, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:19.368940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59016b-4490-4226-8ba8-1c4a31634c53 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:19.383482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 938/1875] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:19 2022] PUT /identity/v3/projects/37593cd9f66b4848b0021e556c6a75cf/users/29b26036a0834f30973fc4501390d3e3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:19.404218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98fc2c-25a5-4521-99d9-62d8fb0378ba None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.404218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98fc2c-25a5-4521-99d9-62d8fb0378ba None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.404218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98fc2c-25a5-4521-99d9-62d8fb0378ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.464764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ae98fc2c-25a5-4521-99d9-62d8fb0378ba None None] MFA Rules not processed for user `29b26036a0834f30973fc4501390d3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:19.466300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ae98fc2c-25a5-4521-99d9-62d8fb0378ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:19.544806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 938/1876] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:19 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:19.556130 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59016b-4490-4226-8ba8-1c4a31634c53 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:19.557999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2e59016b-4490-4226-8ba8-1c4a31634c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:19.618840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:19.647747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59016b-4490-4226-8ba8-1c4a31634c53 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:19.663851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71911ca0-9e4a-4875-9c62-e17048cd2f80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:19.663851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71911ca0-9e4a-4875-9c62-e17048cd2f80 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.663851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71911ca0-9e4a-4875-9c62-e17048cd2f80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.663851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71911ca0-9e4a-4875-9c62-e17048cd2f80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.667573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71911ca0-9e4a-4875-9c62-e17048cd2f80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:19.668964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71911ca0-9e4a-4875-9c62-e17048cd2f80 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:19.679693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98fc2c-25a5-4521-99d9-62d8fb0378ba None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:19.680236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 939/1877] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:19 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:19.682551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ae98fc2c-25a5-4521-99d9-62d8fb0378ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:19.730469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71911ca0-9e4a-4875-9c62-e17048cd2f80 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:19.733067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-71911ca0-9e4a-4875-9c62-e17048cd2f80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:19.749729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71911ca0-9e4a-4875-9c62-e17048cd2f80 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:19.754479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d431955-e4f9-489d-ab8f-90e93fe29ea9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:19.755146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d431955-e4f9-489d-ab8f-90e93fe29ea9 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.755512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d431955-e4f9-489d-ab8f-90e93fe29ea9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.755850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d431955-e4f9-489d-ab8f-90e93fe29ea9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.757812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6d431955-e4f9-489d-ab8f-90e93fe29ea9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:19.777680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98fc2c-25a5-4521-99d9-62d8fb0378ba None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:19.782880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:19.783540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.783820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.784094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] PATH_INFO: `/v3/projects/12b62aa5ee084451a3be1f771a678b69` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.785212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b62aa5ee084451a3be1f771a678b69)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:19.787142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:19.822551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d431955-e4f9-489d-ab8f-90e93fe29ea9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:19.824156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d431955-e4f9-489d-ab8f-90e93fe29ea9 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:19.834281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 940/1878] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:19.836839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b62aa5ee084451a3be1f771a678b69', 'request_id': 'req-6caadb30-3512-4c88-84b5-b7260492a1ea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:19.843880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b62aa5ee084451a3be1f771a678b69', 'request_id': 'req-6caadb30-3512-4c88-84b5-b7260492a1ea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:19.852877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b62aa5ee084451a3be1f771a678b69', 'request_id': 'req-6caadb30-3512-4c88-84b5-b7260492a1ea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:19.867696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] The token cache is being invalidate because project 12b62aa5ee084451a3be1f771a678b69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:19.867982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:19.868635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:19.870180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 939/1879] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:19 2022] DELETE /identity/v3/projects/12b62aa5ee084451a3be1f771a678b69 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:19.874769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d431955-e4f9-489d-ab8f-90e93fe29ea9 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:19.885649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d431955-e4f9-489d-ab8f-90e93fe29ea9 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:19.947384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:19.947994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:19.948574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:19.948832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] PATH_INFO: `/v3/users/0ca2f4c31ff44943982ad35dacc418bf` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:19.950768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ca2f4c31ff44943982ad35dacc418bf)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:19.952770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:20.000080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ca2f4c31ff44943982ad35dacc418bf'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.000785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ca2f4c31ff44943982ad35dacc418bf'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.008145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ca2f4c31ff44943982ad35dacc418bf'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.016224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ca2f4c31ff44943982ad35dacc418bf'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.016902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ca2f4c31ff44943982ad35dacc418bf'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.039254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 941/1880] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:19 2022] DELETE /identity/v3/users/0ca2f4c31ff44943982ad35dacc418bf => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:20.349141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:20.352078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:20.380764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caadb30-3512-4c88-84b5-b7260492a1ea admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:20.385063 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:20.454132 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:20.454692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:20.454972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:20.455243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] PATH_INFO: `/v3/projects/b2c1d30a18e1471999b60c14b685e536` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:20.456403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2c1d30a18e1471999b60c14b685e536)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:20.458414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:20.498768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:20.500369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:20.514823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:20.520423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1964c6d4-b6ef-4a93-997a-e57f784fc6dc admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:20.531893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2c1d30a18e1471999b60c14b685e536', 'request_id': 'req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.538311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2c1d30a18e1471999b60c14b685e536', 'request_id': 'req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.548110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2c1d30a18e1471999b60c14b685e536', 'request_id': 'req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.566826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] The token cache is being invalidate because project b2c1d30a18e1471999b60c14b685e536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:20.567128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.580651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fa0f7bef-c050-4f08-b45c-f068a5ddf8b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.582415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 940/1881] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:20 2022] DELETE /identity/v3/projects/b2c1d30a18e1471999b60c14b685e536 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:20.587675 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc1361-96bd-451d-b011-2f958db5e163 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:20.588474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc1361-96bd-451d-b011-2f958db5e163 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:20.588474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bc1361-96bd-451d-b011-2f958db5e163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:20.620250 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-70bc1361-96bd-451d-b011-2f958db5e163 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:20.623492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-70bc1361-96bd-451d-b011-2f958db5e163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:20.624145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:20.624937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:20.625129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:20.625629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] PATH_INFO: `/v3/users/f79e3ae5e01d4e35b46dffdb48798e24` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:20.626945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f79e3ae5e01d4e35b46dffdb48798e24)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:20.628934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:20.651597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 941/1882] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:20.691142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aba9fc-f303-4737-8b60-4d411c460a90 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:20.692614 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aba9fc-f303-4737-8b60-4d411c460a90 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:20.692614 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aba9fc-f303-4737-8b60-4d411c460a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:20.728355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d7aba9fc-f303-4737-8b60-4d411c460a90 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:20.729588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d7aba9fc-f303-4737-8b60-4d411c460a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:20.750154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 942/1883] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:20.752562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aba9fc-f303-4737-8b60-4d411c460a90 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:20.759661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f79e3ae5e01d4e35b46dffdb48798e24'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.762055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f79e3ae5e01d4e35b46dffdb48798e24'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.780157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aba9fc-f303-4737-8b60-4d411c460a90 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:20.784698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce6eec-66f7-4c88-bc54-0c5c8f7b6f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:20.790518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6eec-66f7-4c88-bc54-0c5c8f7b6f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:20.790518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6eec-66f7-4c88-bc54-0c5c8f7b6f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:20.790518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce6eec-66f7-4c88-bc54-0c5c8f7b6f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:20.790518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce6eec-66f7-4c88-bc54-0c5c8f7b6f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:20.790518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce6eec-66f7-4c88-bc54-0c5c8f7b6f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:20.792827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f79e3ae5e01d4e35b46dffdb48798e24'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.793749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fce6eec-66f7-4c88-bc54-0c5c8f7b6f16 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:20.804274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f79e3ae5e01d4e35b46dffdb48798e24'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.805420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-fa3cc8ad-b1fd-49f9-9662-53bf768730b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f79e3ae5e01d4e35b46dffdb48798e24'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:20.814653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 943/1884] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:20.819647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce6eec-66f7-4c88-bc54-0c5c8f7b6f16 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:20.827067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2fce6eec-66f7-4c88-bc54-0c5c8f7b6f16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:20.828947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 942/1885] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:20 2022] DELETE /identity/v3/users/f79e3ae5e01d4e35b46dffdb48798e24 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:20.833529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1661f5-cf37-41a8-9a77-542bbf83f717 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:20.833812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1661f5-cf37-41a8-9a77-542bbf83f717 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:20.834083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1661f5-cf37-41a8-9a77-542bbf83f717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:20.847735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fce6eec-66f7-4c88-bc54-0c5c8f7b6f16 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:20.862994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-5f1661f5-cf37-41a8-9a77-542bbf83f717 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:20.864512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5f1661f5-cf37-41a8-9a77-542bbf83f717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:20.944960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d1b71-843f-4f09-bc02-95612f7269f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:20.945638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d1b71-843f-4f09-bc02-95612f7269f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:20.946043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d1b71-843f-4f09-bc02-95612f7269f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:20.946364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d1b71-843f-4f09-bc02-95612f7269f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:20.947229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d1b71-843f-4f09-bc02-95612f7269f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:20.950550 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039d1b71-843f-4f09-bc02-95612f7269f5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:20.952612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 943/1886] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:20.976649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:20.993139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 944/1887] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:20 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:20.997848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1661f5-cf37-41a8-9a77-542bbf83f717 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:21.008048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1661f5-cf37-41a8-9a77-542bbf83f717 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:21.032720 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:21.033596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d1b71-843f-4f09-bc02-95612f7269f5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:21.036833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-039d1b71-843f-4f09-bc02-95612f7269f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:21.131702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70469d-12ed-441c-9efa-01aea980e1bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:21.134033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039d1b71-843f-4f09-bc02-95612f7269f5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:21.136398 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70469d-12ed-441c-9efa-01aea980e1bc admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:21.138289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70469d-12ed-441c-9efa-01aea980e1bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:21.138768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee70469d-12ed-441c-9efa-01aea980e1bc admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:21.139616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d920b-88f2-443d-8cee-d263c0c091f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:21.140475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d920b-88f2-443d-8cee-d263c0c091f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:21.140587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d920b-88f2-443d-8cee-d263c0c091f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:21.140960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256d920b-88f2-443d-8cee-d263c0c091f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:21.141284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70469d-12ed-441c-9efa-01aea980e1bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:21.144795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee70469d-12ed-441c-9efa-01aea980e1bc admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:21.158596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d920b-88f2-443d-8cee-d263c0c091f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:21.162213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256d920b-88f2-443d-8cee-d263c0c091f2 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:21.173632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 944/1888] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:20 2022] POST /identity/v3/users => generated 287 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:21.176190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70469d-12ed-441c-9efa-01aea980e1bc admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:21.182243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 945/1889] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:21.190988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee70469d-12ed-441c-9efa-01aea980e1bc admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:21.196684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25919ebb-06af-40da-8ef6-5240a4fa8db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:21.197737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25919ebb-06af-40da-8ef6-5240a4fa8db7 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:21.198337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25919ebb-06af-40da-8ef6-5240a4fa8db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:21.198676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25919ebb-06af-40da-8ef6-5240a4fa8db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:21.199414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25919ebb-06af-40da-8ef6-5240a4fa8db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:21.201601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25919ebb-06af-40da-8ef6-5240a4fa8db7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:21.213004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 945/1890] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:21.217729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d920b-88f2-443d-8cee-d263c0c091f2 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:21.235493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256d920b-88f2-443d-8cee-d263c0c091f2 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:21.240885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7985575d-76de-4dca-851e-a70168f7dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:21.242731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985575d-76de-4dca-851e-a70168f7dc43 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:21.243020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985575d-76de-4dca-851e-a70168f7dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:21.243359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7985575d-76de-4dca-851e-a70168f7dc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:21.244092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985575d-76de-4dca-851e-a70168f7dc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:21.246357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7985575d-76de-4dca-851e-a70168f7dc43 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:21.263967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 946/1891] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:21.268113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25919ebb-06af-40da-8ef6-5240a4fa8db7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:21.281024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25919ebb-06af-40da-8ef6-5240a4fa8db7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:21.304803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adddfc7-d6bf-48ef-b5c8-2bb763304834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:21.306691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adddfc7-d6bf-48ef-b5c8-2bb763304834 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:21.307862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adddfc7-d6bf-48ef-b5c8-2bb763304834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:21.309964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adddfc7-d6bf-48ef-b5c8-2bb763304834 admin admin] PATH_INFO: `/v3/projects/c9e09828a7e642618d6dd60e3bbaeb03/users/fbda199975b04fa190689f1ac1d0cdff/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:21.363663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adddfc7-d6bf-48ef-b5c8-2bb763304834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9e09828a7e642618d6dd60e3bbaeb03, user_id=fbda199975b04fa190689f1ac1d0cdff, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:21.366506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adddfc7-d6bf-48ef-b5c8-2bb763304834 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:21.390226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 946/1892] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:21 2022] PUT /identity/v3/projects/c9e09828a7e642618d6dd60e3bbaeb03/users/fbda199975b04fa190689f1ac1d0cdff/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:21.396038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61445f-ea91-479b-a65f-540fad7bdb4b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:21.396472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61445f-ea91-479b-a65f-540fad7bdb4b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:21.396754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61445f-ea91-479b-a65f-540fad7bdb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:21.421162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6c61445f-ea91-479b-a65f-540fad7bdb4b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:21.427558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6c61445f-ea91-479b-a65f-540fad7bdb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:21.529773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 947/1893] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:21.555793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adddfc7-d6bf-48ef-b5c8-2bb763304834 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:21.687020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adddfc7-d6bf-48ef-b5c8-2bb763304834 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:21.706644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0016e-7fc9-4d3b-9260-c0d9cf277ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:21.716457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0016e-7fc9-4d3b-9260-c0d9cf277ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:21.716742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0016e-7fc9-4d3b-9260-c0d9cf277ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:21.717017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e0016e-7fc9-4d3b-9260-c0d9cf277ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:21.727342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0016e-7fc9-4d3b-9260-c0d9cf277ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:21.728866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e0016e-7fc9-4d3b-9260-c0d9cf277ba9 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:21.741448 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 947/1894] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:22.660294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61445f-ea91-479b-a65f-540fad7bdb4b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:22.662987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6c61445f-ea91-479b-a65f-540fad7bdb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:22.685149 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61445f-ea91-479b-a65f-540fad7bdb4b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:22.691508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:22.692471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:22.692947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:22.693436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] PATH_INFO: `/v3/projects/f7a1837eb73444529dfe072a69d7d670` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:22.695005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a1837eb73444529dfe072a69d7d670)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:22.698083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:22.759874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a1837eb73444529dfe072a69d7d670', 'request_id': 'req-c042f755-8869-4459-aaf3-f5be38bce517'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:22.766717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a1837eb73444529dfe072a69d7d670', 'request_id': 'req-c042f755-8869-4459-aaf3-f5be38bce517'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:22.781605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a1837eb73444529dfe072a69d7d670', 'request_id': 'req-c042f755-8869-4459-aaf3-f5be38bce517'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:22.796420 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] The token cache is being invalidate because project f7a1837eb73444529dfe072a69d7d670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:22.796962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:22.797798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:22.800013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 948/1895] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:22 2022] DELETE /identity/v3/projects/f7a1837eb73444529dfe072a69d7d670 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:22.804431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0016e-7fc9-4d3b-9260-c0d9cf277ba9 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:22.815345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e0016e-7fc9-4d3b-9260-c0d9cf277ba9 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:22.884353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:22.885439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:22.886559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:22.886989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] PATH_INFO: `/v3/users/7bceb3abc4ca4e64b0e2964de1f01233` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:22.889391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bceb3abc4ca4e64b0e2964de1f01233)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:22.891710 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:22.928997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:22.969266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bceb3abc4ca4e64b0e2964de1f01233'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:22.970075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bceb3abc4ca4e64b0e2964de1f01233'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:22.981806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bceb3abc4ca4e64b0e2964de1f01233'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:22.991622 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bceb3abc4ca4e64b0e2964de1f01233'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:22.992540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9478ead2-ba61-4dff-b8ce-4fc303060fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bceb3abc4ca4e64b0e2964de1f01233'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.021325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 948/1896] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:22 2022] DELETE /identity/v3/users/7bceb3abc4ca4e64b0e2964de1f01233 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:23.036323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:23.038303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:23.058720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c042f755-8869-4459-aaf3-f5be38bce517 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:23.070633 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:23.138890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:23.139757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.139907 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.140156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] PATH_INFO: `/v3/users/b2da365f5d5346db95a21732fe68161f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.141285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2da365f5d5346db95a21732fe68161f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:23.143306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:23.208835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2da365f5d5346db95a21732fe68161f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.220382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2da365f5d5346db95a21732fe68161f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.222185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2da365f5d5346db95a21732fe68161f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.222702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2da365f5d5346db95a21732fe68161f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.234817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-aeb70b58-cb1b-4982-9997-77df96dffa8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2da365f5d5346db95a21732fe68161f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.258407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 949/1897] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:23 2022] DELETE /identity/v3/users/b2da365f5d5346db95a21732fe68161f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:23.264092 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1852-832f-4225-a117-83d3ca4651d2 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.266386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1852-832f-4225-a117-83d3ca4651d2 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.267326 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1852-832f-4225-a117-83d3ca4651d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.290547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365e891-3cce-4682-9297-9fbca4730581 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.291018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365e891-3cce-4682-9297-9fbca4730581 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.291476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365e891-3cce-4682-9297-9fbca4730581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.313431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-67aa1852-832f-4225-a117-83d3ca4651d2 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:23.315101 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-67aa1852-832f-4225-a117-83d3ca4651d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:23.337787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e365e891-3cce-4682-9297-9fbca4730581 None None] MFA Rules not processed for user `fbda199975b04fa190689f1ac1d0cdff`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:23.340187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e365e891-3cce-4682-9297-9fbca4730581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:23.424260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 949/1898] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:23.432188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa1852-832f-4225-a117-83d3ca4651d2 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:23.440227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-67aa1852-832f-4225-a117-83d3ca4651d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:23.454842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa1852-832f-4225-a117-83d3ca4651d2 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:23.459970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:23.460933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.461324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.461787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] PATH_INFO: `/v3/projects/33d2d40c9ef54d3394886dffae08c253` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.463778 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d2d40c9ef54d3394886dffae08c253)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:23.467163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:23.473232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 950/1899] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:23.479997 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365e891-3cce-4682-9297-9fbca4730581 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:23.483176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e365e891-3cce-4682-9297-9fbca4730581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:23.532314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:23.541380 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d2d40c9ef54d3394886dffae08c253', 'request_id': 'req-0efd77ce-195c-4139-9573-4c68b6aeba58'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.558712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d2d40c9ef54d3394886dffae08c253', 'request_id': 'req-0efd77ce-195c-4139-9573-4c68b6aeba58'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.571397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d2d40c9ef54d3394886dffae08c253', 'request_id': 'req-0efd77ce-195c-4139-9573-4c68b6aeba58'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.591077 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] The token cache is being invalidate because project 33d2d40c9ef54d3394886dffae08c253 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:23.592041 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.597094 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:23.599498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 950/1900] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:23 2022] DELETE /identity/v3/projects/33d2d40c9ef54d3394886dffae08c253 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:23.600449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:23.603549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:23.610800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365e891-3cce-4682-9297-9fbca4730581 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:23.654040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:23.701957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba532ee-40b1-4089-ac9a-a13057539492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:23.703327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba532ee-40b1-4089-ac9a-a13057539492 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.703836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba532ee-40b1-4089-ac9a-a13057539492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.704305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba532ee-40b1-4089-ac9a-a13057539492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.706641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2ba532ee-40b1-4089-ac9a-a13057539492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:23.723960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd77ce-195c-4139-9573-4c68b6aeba58 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:23.729972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be1556f-130b-4fc8-a72e-3a55c8f379fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:23.730713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1556f-130b-4fc8-a72e-3a55c8f379fc admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.731901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1556f-130b-4fc8-a72e-3a55c8f379fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.732051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be1556f-130b-4fc8-a72e-3a55c8f379fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.733155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1556f-130b-4fc8-a72e-3a55c8f379fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:23.735726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be1556f-130b-4fc8-a72e-3a55c8f379fc admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:23.772344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 951/1901] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:23 2022] POST /identity/v3/projects => generated 373 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:23.776402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6a24-9795-4b00-a93a-72e625b29573 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.776402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6a24-9795-4b00-a93a-72e625b29573 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.776402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ae6a24-9795-4b00-a93a-72e625b29573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.798970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-08ae6a24-9795-4b00-a93a-72e625b29573 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:23.800460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba532ee-40b1-4089-ac9a-a13057539492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:23.800579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-08ae6a24-9795-4b00-a93a-72e625b29573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:23.802183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba532ee-40b1-4089-ac9a-a13057539492 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:23.817747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 951/1902] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:23.818992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba532ee-40b1-4089-ac9a-a13057539492 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:23.831693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba532ee-40b1-4089-ac9a-a13057539492 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:23.889204 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 952/1903] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:23.891785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573b530-f56c-48a4-baa4-87babbe5458d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.892073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573b530-f56c-48a4-baa4-87babbe5458d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.893062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a573b530-f56c-48a4-baa4-87babbe5458d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.908220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa7ae6-59ac-4997-b1e5-9fa7656c45d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:23.909016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa7ae6-59ac-4997-b1e5-9fa7656c45d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.909305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa7ae6-59ac-4997-b1e5-9fa7656c45d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.909583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa7ae6-59ac-4997-b1e5-9fa7656c45d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.910240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa7ae6-59ac-4997-b1e5-9fa7656c45d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:23.912713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa7ae6-59ac-4997-b1e5-9fa7656c45d1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:23.920153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a573b530-f56c-48a4-baa4-87babbe5458d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:23.921714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a573b530-f56c-48a4-baa4-87babbe5458d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:23.931928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 953/1904] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:23.941465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 952/1905] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:23 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:23.946189 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa7ae6-59ac-4997-b1e5-9fa7656c45d1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:23.947758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573b530-f56c-48a4-baa4-87babbe5458d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:23.959578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa7ae6-59ac-4997-b1e5-9fa7656c45d1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:23.970232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10686b0e-c4ec-40dd-a2b3-e93cab06a9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:23.970879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686b0e-c4ec-40dd-a2b3-e93cab06a9fe admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.971155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686b0e-c4ec-40dd-a2b3-e93cab06a9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.971423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10686b0e-c4ec-40dd-a2b3-e93cab06a9fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.972092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10686b0e-c4ec-40dd-a2b3-e93cab06a9fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:23.972805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a573b530-f56c-48a4-baa4-87babbe5458d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:23.973705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10686b0e-c4ec-40dd-a2b3-e93cab06a9fe admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:23.978249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1849c-8ca6-43ca-b544-42f29e9200b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:23.979172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1849c-8ca6-43ca-b544-42f29e9200b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:23.979661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1849c-8ca6-43ca-b544-42f29e9200b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:23.980095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f1849c-8ca6-43ca-b544-42f29e9200b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:23.982040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1849c-8ca6-43ca-b544-42f29e9200b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:23.982425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1849c-8ca6-43ca-b544-42f29e9200b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:23.984102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f1849c-8ca6-43ca-b544-42f29e9200b7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:23.990722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 953/1906] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:23.994693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10686b0e-c4ec-40dd-a2b3-e93cab06a9fe admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:23.997141 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 954/1907] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:24.002309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:24.003185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1849c-8ca6-43ca-b544-42f29e9200b7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.007144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10686b0e-c4ec-40dd-a2b3-e93cab06a9fe admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.014316 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d8447-a720-44f9-855b-37178ef8f446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.015060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d8447-a720-44f9-855b-37178ef8f446 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.015253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f1849c-8ca6-43ca-b544-42f29e9200b7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.015356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d8447-a720-44f9-855b-37178ef8f446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.015630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2d8447-a720-44f9-855b-37178ef8f446 admin admin] PATH_INFO: `/v3/projects/b06b34af70ca40328ec4480d4558aec7/users/226c04e7d4ef407d829867bcc77e38f9/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.021702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939d84e-f1b3-4630-8a77-a064deba9f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.022370 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d84e-f1b3-4630-8a77-a064deba9f62 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.022651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d84e-f1b3-4630-8a77-a064deba9f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.022919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6939d84e-f1b3-4630-8a77-a064deba9f62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.023674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939d84e-f1b3-4630-8a77-a064deba9f62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.025741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6939d84e-f1b3-4630-8a77-a064deba9f62 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.037489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d8447-a720-44f9-855b-37178ef8f446 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06b34af70ca40328ec4480d4558aec7, user_id=226c04e7d4ef407d829867bcc77e38f9, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.040472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2d8447-a720-44f9-855b-37178ef8f446 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.054622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 954/1908] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:23 2022] PUT /identity/v3/projects/b06b34af70ca40328ec4480d4558aec7/users/226c04e7d4ef407d829867bcc77e38f9/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:24.057592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 955/1909] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:24 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:24.058610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d8447-a720-44f9-855b-37178ef8f446 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.061520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939d84e-f1b3-4630-8a77-a064deba9f62 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.130312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2d8447-a720-44f9-855b-37178ef8f446 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.135080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73389398-2d21-40f0-acb3-0dd1f6b99403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.135751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73389398-2d21-40f0-acb3-0dd1f6b99403 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.135860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6939d84e-f1b3-4630-8a77-a064deba9f62 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.136056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73389398-2d21-40f0-acb3-0dd1f6b99403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.136301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73389398-2d21-40f0-acb3-0dd1f6b99403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.136947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73389398-2d21-40f0-acb3-0dd1f6b99403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.139127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73389398-2d21-40f0-acb3-0dd1f6b99403 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.141693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b815cb-ff48-4af7-bf6d-005fac3e03e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.142440 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b815cb-ff48-4af7-bf6d-005fac3e03e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.142775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b815cb-ff48-4af7-bf6d-005fac3e03e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.143104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b815cb-ff48-4af7-bf6d-005fac3e03e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.143838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b815cb-ff48-4af7-bf6d-005fac3e03e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.145809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b815cb-ff48-4af7-bf6d-005fac3e03e3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.153936 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 955/1910] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:24.155237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73389398-2d21-40f0-acb3-0dd1f6b99403 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.164196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:24.169025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73389398-2d21-40f0-acb3-0dd1f6b99403 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.173113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 956/1911] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:24 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:24.176807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b815cb-ff48-4af7-bf6d-005fac3e03e3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.178435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312ee54-4710-41e0-bd33-b7997ca81184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.179206 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ee54-4710-41e0-bd33-b7997ca81184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.179621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ee54-4710-41e0-bd33-b7997ca81184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.179996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e312ee54-4710-41e0-bd33-b7997ca81184 admin admin] PATH_INFO: `/v3/projects/b06b34af70ca40328ec4480d4558aec7/users/226c04e7d4ef407d829867bcc77e38f9/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.182356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312ee54-4710-41e0-bd33-b7997ca81184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06b34af70ca40328ec4480d4558aec7, user_id=226c04e7d4ef407d829867bcc77e38f9, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.185496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e312ee54-4710-41e0-bd33-b7997ca81184 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.187267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b815cb-ff48-4af7-bf6d-005fac3e03e3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.192464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd354b-b0a9-4608-8986-3422504c5c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.193075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd354b-b0a9-4608-8986-3422504c5c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.193351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd354b-b0a9-4608-8986-3422504c5c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.193924 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd354b-b0a9-4608-8986-3422504c5c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.194286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd354b-b0a9-4608-8986-3422504c5c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.195935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd354b-b0a9-4608-8986-3422504c5c8a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.196951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 956/1912] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:24 2022] PUT /identity/v3/projects/b06b34af70ca40328ec4480d4558aec7/users/226c04e7d4ef407d829867bcc77e38f9/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:24.210395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 957/1913] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:24.211793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312ee54-4710-41e0-bd33-b7997ca81184 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.240901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd354b-b0a9-4608-8986-3422504c5c8a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.243763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e8fd354b-b0a9-4608-8986-3422504c5c8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:24.268818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd354b-b0a9-4608-8986-3422504c5c8a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.305666 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e312ee54-4710-41e0-bd33-b7997ca81184 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.313531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508421cd-fe3b-4a87-a23b-1cf49ee1dede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.314126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508421cd-fe3b-4a87-a23b-1cf49ee1dede admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.314405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508421cd-fe3b-4a87-a23b-1cf49ee1dede admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.314671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508421cd-fe3b-4a87-a23b-1cf49ee1dede admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.315322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508421cd-fe3b-4a87-a23b-1cf49ee1dede admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.317128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508421cd-fe3b-4a87-a23b-1cf49ee1dede admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.337071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 957/1914] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:24.338229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508421cd-fe3b-4a87-a23b-1cf49ee1dede admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.351526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508421cd-fe3b-4a87-a23b-1cf49ee1dede admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.359131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e64556c-1b4e-4479-b56c-fb293d60c42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.360309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64556c-1b4e-4479-b56c-fb293d60c42b admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.360919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64556c-1b4e-4479-b56c-fb293d60c42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.361578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e64556c-1b4e-4479-b56c-fb293d60c42b admin admin] PATH_INFO: `/v3/projects/8bf2245b0b9147bfb06a9ab8dad145af/users/4db9ae3f2548400da34c2535abcbe3e9/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.373517 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.374188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.374494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.374739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] PATH_INFO: `/v3/projects/6cb913ede30047cdb38d4c524885b376` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.376172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cb913ede30047cdb38d4c524885b376)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.378932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.380527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64556c-1b4e-4479-b56c-fb293d60c42b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bf2245b0b9147bfb06a9ab8dad145af, user_id=4db9ae3f2548400da34c2535abcbe3e9, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.383277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e64556c-1b4e-4479-b56c-fb293d60c42b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.397811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 958/1915] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:24 2022] PUT /identity/v3/projects/8bf2245b0b9147bfb06a9ab8dad145af/users/4db9ae3f2548400da34c2535abcbe3e9/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:24.410148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dc565-66c1-42e5-bac8-ebab60e5fdaa None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.410644 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dc565-66c1-42e5-bac8-ebab60e5fdaa None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.410917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394dc565-66c1-42e5-bac8-ebab60e5fdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.458542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cb913ede30047cdb38d4c524885b376', 'request_id': 'req-2badf38f-7c51-4897-a907-6a5023854040'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:24.472530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-394dc565-66c1-42e5-bac8-ebab60e5fdaa None None] MFA Rules not processed for user `4db9ae3f2548400da34c2535abcbe3e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:24.473771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-394dc565-66c1-42e5-bac8-ebab60e5fdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:24.476284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cb913ede30047cdb38d4c524885b376', 'request_id': 'req-2badf38f-7c51-4897-a907-6a5023854040'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:24.498017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cb913ede30047cdb38d4c524885b376', 'request_id': 'req-2badf38f-7c51-4897-a907-6a5023854040'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:24.518884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] The token cache is being invalidate because project 6cb913ede30047cdb38d4c524885b376 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:24.519233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:24.520344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:24.521892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 958/1916] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:24 2022] DELETE /identity/v3/projects/6cb913ede30047cdb38d4c524885b376 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:24.523982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:24.526669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.529509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:24.588654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 959/1917] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:24.593703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dc565-66c1-42e5-bac8-ebab60e5fdaa None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.595825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-394dc565-66c1-42e5-bac8-ebab60e5fdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:24.629592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2badf38f-7c51-4897-a907-6a5023854040 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.636409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.637121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.637400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.637677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] PATH_INFO: `/v3/users/cc7124871914409e8d3aaefc0eb183fa` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.639040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7124871914409e8d3aaefc0eb183fa)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.641809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.671551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:24.707421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7124871914409e8d3aaefc0eb183fa'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:24.708264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7124871914409e8d3aaefc0eb183fa'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:24.719128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7124871914409e8d3aaefc0eb183fa'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:24.720283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394dc565-66c1-42e5-bac8-ebab60e5fdaa None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.725944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282ac13-fc7f-43ff-bf19-fe94d3286ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.727219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282ac13-fc7f-43ff-bf19-fe94d3286ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.727527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282ac13-fc7f-43ff-bf19-fe94d3286ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.727800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282ac13-fc7f-43ff-bf19-fe94d3286ad1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.728440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282ac13-fc7f-43ff-bf19-fe94d3286ad1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.729123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7124871914409e8d3aaefc0eb183fa'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:24.729926 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d1a5ae54-351e-4298-a385-6df4aae75914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7124871914409e8d3aaefc0eb183fa'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:24.730788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6282ac13-fc7f-43ff-bf19-fe94d3286ad1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.769922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 959/1918] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:24 2022] DELETE /identity/v3/users/cc7124871914409e8d3aaefc0eb183fa => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:24.783895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 960/1919] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:24 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:24.783895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282ac13-fc7f-43ff-bf19-fe94d3286ad1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.783895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6282ac13-fc7f-43ff-bf19-fe94d3286ad1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:24.784150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dbe5e-76fe-4b35-a730-59efbda71962 None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.784150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dbe5e-76fe-4b35-a730-59efbda71962 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.784150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17dbe5e-76fe-4b35-a730-59efbda71962 None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.785402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 960/1920] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:14:24 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:14:24.799651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea62f0-e79d-4781-83da-949d459413e2 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.799981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea62f0-e79d-4781-83da-949d459413e2 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.800266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea62f0-e79d-4781-83da-949d459413e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.845723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-5bea62f0-e79d-4781-83da-949d459413e2 None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:24.846944 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5bea62f0-e79d-4781-83da-949d459413e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:24.868647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6282ac13-fc7f-43ff-bf19-fe94d3286ad1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.875841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf849b8-054b-492a-9b4f-d877264e838d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.876672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf849b8-054b-492a-9b4f-d877264e838d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.876950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf849b8-054b-492a-9b4f-d877264e838d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.877224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf849b8-054b-492a-9b4f-d877264e838d admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.877995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf849b8-054b-492a-9b4f-d877264e838d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.880152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf849b8-054b-492a-9b4f-d877264e838d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.926215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 961/1921] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:24 2022] POST /identity/v3/users => generated 299 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:24.927574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf849b8-054b-492a-9b4f-d877264e838d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.939833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf849b8-054b-492a-9b4f-d877264e838d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.951750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d9788-606c-4154-b20a-63a02faff4b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.952375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9788-606c-4154-b20a-63a02faff4b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.952649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9788-606c-4154-b20a-63a02faff4b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.952918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d9788-606c-4154-b20a-63a02faff4b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:24.953732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d9788-606c-4154-b20a-63a02faff4b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:24.955709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8d9788-606c-4154-b20a-63a02faff4b9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:24.959065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 961/1922] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:14:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:24.971506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 962/1923] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:24.977770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea62f0-e79d-4781-83da-949d459413e2 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.978008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d9788-606c-4154-b20a-63a02faff4b9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:24.978112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5bea62f0-e79d-4781-83da-949d459413e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:24.990498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d9788-606c-4154-b20a-63a02faff4b9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.995421 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce29f84-8b91-4613-93a8-1160cb6b74e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:24.996117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce29f84-8b91-4613-93a8-1160cb6b74e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:24.997062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce29f84-8b91-4613-93a8-1160cb6b74e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:24.997199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea62f0-e79d-4781-83da-949d459413e2 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:24.997769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce29f84-8b91-4613-93a8-1160cb6b74e7 admin admin] PATH_INFO: `/v3/projects/887c0b2bf0e54ee18c28ee1e90430efd/users/051e339881004ce7821a513bb4b11798/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:25.003034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119d4ca-6218-42d6-822d-d54b802d1f6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:25.003684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119d4ca-6218-42d6-822d-d54b802d1f6d service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:25.003985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119d4ca-6218-42d6-822d-d54b802d1f6d service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:25.004264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119d4ca-6218-42d6-822d-d54b802d1f6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:25.014607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119d4ca-6218-42d6-822d-d54b802d1f6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:25.015399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce29f84-8b91-4613-93a8-1160cb6b74e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=887c0b2bf0e54ee18c28ee1e90430efd, user_id=051e339881004ce7821a513bb4b11798, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:25.016330 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119d4ca-6218-42d6-822d-d54b802d1f6d service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:25.018556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce29f84-8b91-4613-93a8-1160cb6b74e7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:25.028638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 962/1924] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:25.039326 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 963/1925] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:24 2022] PUT /identity/v3/projects/887c0b2bf0e54ee18c28ee1e90430efd/users/051e339881004ce7821a513bb4b11798/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:25.040501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119d4ca-6218-42d6-822d-d54b802d1f6d service nova] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:25.052007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119d4ca-6218-42d6-822d-d54b802d1f6d service nova] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:25.119403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549897a5-c808-4f04-a5a9-81f5ed09028f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:25.119975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549897a5-c808-4f04-a5a9-81f5ed09028f admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:25.120255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549897a5-c808-4f04-a5a9-81f5ed09028f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:25.120529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549897a5-c808-4f04-a5a9-81f5ed09028f admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:25.121239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549897a5-c808-4f04-a5a9-81f5ed09028f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:25.125109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549897a5-c808-4f04-a5a9-81f5ed09028f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:25.137842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 963/1926] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:25.141513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce29f84-8b91-4613-93a8-1160cb6b74e7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:25.148762 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:25.154541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce29f84-8b91-4613-93a8-1160cb6b74e7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:25.162547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b935e80-ac3e-410c-bfa3-953284ebbd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:25.163216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b935e80-ac3e-410c-bfa3-953284ebbd43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:25.163551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b935e80-ac3e-410c-bfa3-953284ebbd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:25.163829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b935e80-ac3e-410c-bfa3-953284ebbd43 admin admin] PATH_INFO: `/v3/projects/887c0b2bf0e54ee18c28ee1e90430efd/users/051e339881004ce7821a513bb4b11798/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:25.165822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b935e80-ac3e-410c-bfa3-953284ebbd43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=887c0b2bf0e54ee18c28ee1e90430efd, user_id=051e339881004ce7821a513bb4b11798, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:25.168568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b935e80-ac3e-410c-bfa3-953284ebbd43 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:25.181088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 964/1927] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:25 2022] PUT /identity/v3/projects/887c0b2bf0e54ee18c28ee1e90430efd/users/051e339881004ce7821a513bb4b11798/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:25.193150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094bc2a-e111-4e6d-b3a5-48273853dbc2 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:25.193535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094bc2a-e111-4e6d-b3a5-48273853dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:25.193890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e094bc2a-e111-4e6d-b3a5-48273853dbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:25.243836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:25.263514 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e094bc2a-e111-4e6d-b3a5-48273853dbc2 None None] MFA Rules not processed for user `051e339881004ce7821a513bb4b11798`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:25.265713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e094bc2a-e111-4e6d-b3a5-48273853dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:25.360011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 964/1928] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:25.372382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b935e80-ac3e-410c-bfa3-953284ebbd43 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:25.374143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9b935e80-ac3e-410c-bfa3-953284ebbd43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:25.477822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b935e80-ac3e-410c-bfa3-953284ebbd43 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:25.482327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16cca5-59ae-4666-8a96-93a3f8e0a970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:25.482930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16cca5-59ae-4666-8a96-93a3f8e0a970 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:25.483205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16cca5-59ae-4666-8a96-93a3f8e0a970 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:25.483528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16cca5-59ae-4666-8a96-93a3f8e0a970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:25.492960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16cca5-59ae-4666-8a96-93a3f8e0a970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:25.494424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16cca5-59ae-4666-8a96-93a3f8e0a970 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:25.504277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 965/1929] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:25.654194 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094bc2a-e111-4e6d-b3a5-48273853dbc2 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:25.655681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e094bc2a-e111-4e6d-b3a5-48273853dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:25.735039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef92af-6c40-429c-97e7-497775e51963 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:25.735622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef92af-6c40-429c-97e7-497775e51963 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:25.735622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ef92af-6c40-429c-97e7-497775e51963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:25.769528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-34ef92af-6c40-429c-97e7-497775e51963 None None] MFA Rules not processed for user `226c04e7d4ef407d829867bcc77e38f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:25.770449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-34ef92af-6c40-429c-97e7-497775e51963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:25.771286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e094bc2a-e111-4e6d-b3a5-48273853dbc2 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:25.782028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:25.782235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:25.782754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:25.782997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] PATH_INFO: `/v3/projects/942d783d585c484985d0f43f36360d42` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:25.788178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=942d783d585c484985d0f43f36360d42)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:25.790298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:25.881702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '942d783d585c484985d0f43f36360d42', 'request_id': 'req-90c31e5d-7765-47e0-894d-85e843c900b7'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:25.892842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '942d783d585c484985d0f43f36360d42', 'request_id': 'req-90c31e5d-7765-47e0-894d-85e843c900b7'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:25.900456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '942d783d585c484985d0f43f36360d42', 'request_id': 'req-90c31e5d-7765-47e0-894d-85e843c900b7'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:25.910189 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 966/1930] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:25 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:25.922605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef92af-6c40-429c-97e7-497775e51963 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:25.924180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-34ef92af-6c40-429c-97e7-497775e51963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:25.941790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ef92af-6c40-429c-97e7-497775e51963 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:25.942397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] The token cache is being invalidate because project 942d783d585c484985d0f43f36360d42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:25.942758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:25.943675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:25.952376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 965/1931] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:25 2022] DELETE /identity/v3/projects/942d783d585c484985d0f43f36360d42 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:25.956168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:25.957937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:26.001612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:26.035912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239a8d53-d718-462b-802e-661dfefc9299 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:26.036741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a8d53-d718-462b-802e-661dfefc9299 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:26.037099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a8d53-d718-462b-802e-661dfefc9299 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:26.037998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239a8d53-d718-462b-802e-661dfefc9299 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:26.048209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-239a8d53-d718-462b-802e-661dfefc9299 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:26.053105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c31e5d-7765-47e0-894d-85e843c900b7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:26.058304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:26.059111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:26.059454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:26.059926 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] PATH_INFO: `/v3/users/29b26036a0834f30973fc4501390d3e3` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:26.062972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b26036a0834f30973fc4501390d3e3)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:26.066127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:26.137119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b26036a0834f30973fc4501390d3e3'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:26.137966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:26.138479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b26036a0834f30973fc4501390d3e3'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:26.147781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b26036a0834f30973fc4501390d3e3'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:26.158631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b26036a0834f30973fc4501390d3e3'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:26.159470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b26036a0834f30973fc4501390d3e3'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:26.184118 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a8d53-d718-462b-802e-661dfefc9299 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:26.189394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239a8d53-d718-462b-802e-661dfefc9299 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:26.192164 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 966/1932] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:25 2022] DELETE /identity/v3/users/29b26036a0834f30973fc4501390d3e3 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:26.219788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 967/1933] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:26.369288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:26.370919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:26.510961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78c72c-9669-4cc0-9818-16a169264c61 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:26.518063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:26.518888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:26.519175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:26.519467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] PATH_INFO: `/v3/users/4db9ae3f2548400da34c2535abcbe3e9` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:26.520631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db9ae3f2548400da34c2535abcbe3e9)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:26.537672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:26.629233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db9ae3f2548400da34c2535abcbe3e9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:26.630799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db9ae3f2548400da34c2535abcbe3e9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:26.632763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:26.648528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db9ae3f2548400da34c2535abcbe3e9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:26.660829 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db9ae3f2548400da34c2535abcbe3e9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:26.663389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db9ae3f2548400da34c2535abcbe3e9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:26.699332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 967/1934] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:26 2022] DELETE /identity/v3/users/4db9ae3f2548400da34c2535abcbe3e9 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:26.708383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2ddec-ce17-44ce-8c4c-88c102a29c4e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:26.708772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2ddec-ce17-44ce-8c4c-88c102a29c4e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:26.709106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2ddec-ce17-44ce-8c4c-88c102a29c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:26.733421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:26.737028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:26.788390 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c1a2ddec-ce17-44ce-8c4c-88c102a29c4e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:26.797103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c1a2ddec-ce17-44ce-8c4c-88c102a29c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:26.916942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd3b42-2abd-4695-a07f-a7941c656243 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:26.930867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:26.932309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:26.933516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:26.934212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] PATH_INFO: `/v3/projects/37593cd9f66b4848b0021e556c6a75cf` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:26.936607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 968/1935] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:26.939804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37593cd9f66b4848b0021e556c6a75cf)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:26.942861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:26.990066 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2ddec-ce17-44ce-8c4c-88c102a29c4e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:26.992004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c1a2ddec-ce17-44ce-8c4c-88c102a29c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:27.036179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37593cd9f66b4848b0021e556c6a75cf', 'request_id': 'req-ebb9e470-ba70-401a-b827-7fa765a3b3f9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:27.052665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37593cd9f66b4848b0021e556c6a75cf', 'request_id': 'req-ebb9e470-ba70-401a-b827-7fa765a3b3f9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:27.067618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37593cd9f66b4848b0021e556c6a75cf', 'request_id': 'req-ebb9e470-ba70-401a-b827-7fa765a3b3f9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:27.097177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] The token cache is being invalidate because project 37593cd9f66b4848b0021e556c6a75cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:27.097794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:27.100277 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ebb9e470-ba70-401a-b827-7fa765a3b3f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:27.102654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 968/1936] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:26 2022] DELETE /identity/v3/projects/37593cd9f66b4848b0021e556c6a75cf => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:27.105314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2ddec-ce17-44ce-8c4c-88c102a29c4e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:27.109418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427ed9-e9dd-4367-beb7-2b6541dcc7d8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.109699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427ed9-e9dd-4367-beb7-2b6541dcc7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.109971 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02427ed9-e9dd-4367-beb7-2b6541dcc7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.142994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-02427ed9-e9dd-4367-beb7-2b6541dcc7d8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:27.144254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-02427ed9-e9dd-4367-beb7-2b6541dcc7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:27.202821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b19f7-c247-435e-ad16-5943c23fd168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:27.203617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b19f7-c247-435e-ad16-5943c23fd168 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.204003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b19f7-c247-435e-ad16-5943c23fd168 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.204284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702b19f7-c247-435e-ad16-5943c23fd168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.208326 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-702b19f7-c247-435e-ad16-5943c23fd168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:27.234794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:27.259746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 969/1937] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:27.276021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21ff46-cba5-43a9-bacf-192129f7c8e4 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.276021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21ff46-cba5-43a9-bacf-192129f7c8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.276021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca21ff46-cba5-43a9-bacf-192129f7c8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.323282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-ca21ff46-cba5-43a9-bacf-192129f7c8e4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:27.324401 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:27.325377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ca21ff46-cba5-43a9-bacf-192129f7c8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:27.345342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 970/1938] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:27.350697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21ff46-cba5-43a9-bacf-192129f7c8e4 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:27.351712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b19f7-c247-435e-ad16-5943c23fd168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:27.353934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702b19f7-c247-435e-ad16-5943c23fd168 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:27.361426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca21ff46-cba5-43a9-bacf-192129f7c8e4 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:27.367869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a74c-1964-4c4d-97b5-fba28c6828af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:27.368485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a74c-1964-4c4d-97b5-fba28c6828af admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.368899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a74c-1964-4c4d-97b5-fba28c6828af admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.369177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55a74c-1964-4c4d-97b5-fba28c6828af admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.369893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55a74c-1964-4c4d-97b5-fba28c6828af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:27.370198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55a74c-1964-4c4d-97b5-fba28c6828af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:27.371184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 969/1939] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:27.372751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55a74c-1964-4c4d-97b5-fba28c6828af admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:27.385525 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 971/1940] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:27.390554 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b19f7-c247-435e-ad16-5943c23fd168 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:27.400380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702b19f7-c247-435e-ad16-5943c23fd168 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:27.406122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4f70-809d-4eca-bc7b-8e9d9f766ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:27.406724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4f70-809d-4eca-bc7b-8e9d9f766ca5 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.407001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4f70-809d-4eca-bc7b-8e9d9f766ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.407276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c4f70-809d-4eca-bc7b-8e9d9f766ca5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.407939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c4f70-809d-4eca-bc7b-8e9d9f766ca5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:27.409855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c4f70-809d-4eca-bc7b-8e9d9f766ca5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:27.451299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 970/1941] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:27 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:27.455640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a74c-1964-4c4d-97b5-fba28c6828af admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:27.468949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55a74c-1964-4c4d-97b5-fba28c6828af admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:27.545573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e757b5-45e3-4d3d-87f9-5016ad4633ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:27.546299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e757b5-45e3-4d3d-87f9-5016ad4633ff admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.546580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e757b5-45e3-4d3d-87f9-5016ad4633ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.546856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e757b5-45e3-4d3d-87f9-5016ad4633ff admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.547739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e757b5-45e3-4d3d-87f9-5016ad4633ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:27.558044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e757b5-45e3-4d3d-87f9-5016ad4633ff admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:27.632517 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 972/1942] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:27 2022] POST /identity/v3/users => generated 291 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:27.637126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4f70-809d-4eca-bc7b-8e9d9f766ca5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:27.658092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c4f70-809d-4eca-bc7b-8e9d9f766ca5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:27.666743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316980ac-1fe3-4e3f-94e0-50ae233689f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:27.667912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316980ac-1fe3-4e3f-94e0-50ae233689f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.668861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316980ac-1fe3-4e3f-94e0-50ae233689f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.669296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316980ac-1fe3-4e3f-94e0-50ae233689f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.670200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316980ac-1fe3-4e3f-94e0-50ae233689f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:27.672140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316980ac-1fe3-4e3f-94e0-50ae233689f6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:27.690678 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 971/1943] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:27.691364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e757b5-45e3-4d3d-87f9-5016ad4633ff admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:27.704891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e757b5-45e3-4d3d-87f9-5016ad4633ff admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:27.710154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc433f-5434-4dad-8a98-e57bc62aa3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:27.710876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc433f-5434-4dad-8a98-e57bc62aa3c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.711059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc433f-5434-4dad-8a98-e57bc62aa3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.711298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc433f-5434-4dad-8a98-e57bc62aa3c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.711972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc433f-5434-4dad-8a98-e57bc62aa3c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:27.713577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bc433f-5434-4dad-8a98-e57bc62aa3c3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:27.731679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 973/1944] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:27.733110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316980ac-1fe3-4e3f-94e0-50ae233689f6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:27.747612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316980ac-1fe3-4e3f-94e0-50ae233689f6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:27.755378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773224f-830f-4761-93df-02207b5fbc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:27.756399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773224f-830f-4761-93df-02207b5fbc45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.756869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773224f-830f-4761-93df-02207b5fbc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.757400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8773224f-830f-4761-93df-02207b5fbc45 admin admin] PATH_INFO: `/v3/projects/7fedadec09044f8a9bd7277fb57e7276/users/9dda839a764b4fdfb64d267ac383bf18/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.776202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773224f-830f-4761-93df-02207b5fbc45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fedadec09044f8a9bd7277fb57e7276, user_id=9dda839a764b4fdfb64d267ac383bf18, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:27.780807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8773224f-830f-4761-93df-02207b5fbc45 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:27.795980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 972/1945] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:27 2022] PUT /identity/v3/projects/7fedadec09044f8a9bd7277fb57e7276/users/9dda839a764b4fdfb64d267ac383bf18/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:27.806920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7e63-d8b9-41c5-9af4-02865ddc63d1 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.807212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7e63-d8b9-41c5-9af4-02865ddc63d1 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.807510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575c7e63-d8b9-41c5-9af4-02865ddc63d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.863404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-575c7e63-d8b9-41c5-9af4-02865ddc63d1 None None] MFA Rules not processed for user `9dda839a764b4fdfb64d267ac383bf18`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:27.864944 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-575c7e63-d8b9-41c5-9af4-02865ddc63d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:27.865446 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773224f-830f-4761-93df-02207b5fbc45 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:27.867582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8773224f-830f-4761-93df-02207b5fbc45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:27.941440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8773224f-830f-4761-93df-02207b5fbc45 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:27.946161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc08e0-e3ce-483a-8a3d-a93abae862a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:27.946875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc08e0-e3ce-483a-8a3d-a93abae862a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:27.947153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc08e0-e3ce-483a-8a3d-a93abae862a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:27.947428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffc08e0-e3ce-483a-8a3d-a93abae862a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:27.949275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9ffc08e0-e3ce-483a-8a3d-a93abae862a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:27.949890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 974/1946] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:27 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:27.954200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7e63-d8b9-41c5-9af4-02865ddc63d1 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:27.955674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-575c7e63-d8b9-41c5-9af4-02865ddc63d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:28.003458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:28.014067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc08e0-e3ce-483a-8a3d-a93abae862a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.015556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffc08e0-e3ce-483a-8a3d-a93abae862a8 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.023404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575c7e63-d8b9-41c5-9af4-02865ddc63d1 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:28.025764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 973/1947] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:28.026848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc08e0-e3ce-483a-8a3d-a93abae862a8 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:28.029788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:28.030532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.030692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.030961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] PATH_INFO: `/v3/projects/8bf2245b0b9147bfb06a9ab8dad145af` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.032214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bf2245b0b9147bfb06a9ab8dad145af)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.034258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.037270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffc08e0-e3ce-483a-8a3d-a93abae862a8 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:28.041792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f8e5f-b98d-4c61-a594-3c7029c207f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:28.042517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f8e5f-b98d-4c61-a594-3c7029c207f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.043931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f8e5f-b98d-4c61-a594-3c7029c207f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.044218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f8e5f-b98d-4c61-a594-3c7029c207f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.044855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f8e5f-b98d-4c61-a594-3c7029c207f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.046982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f8e5f-b98d-4c61-a594-3c7029c207f1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.085330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 974/1948] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:28 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:28.089550 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f8e5f-b98d-4c61-a594-3c7029c207f1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:28.098447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bf2245b0b9147bfb06a9ab8dad145af', 'request_id': 'req-cabd5a2d-a91a-4839-8479-6b8d731f0161'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:28.104358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:28.107645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bf2245b0b9147bfb06a9ab8dad145af', 'request_id': 'req-cabd5a2d-a91a-4839-8479-6b8d731f0161'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:28.115077 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bf2245b0b9147bfb06a9ab8dad145af', 'request_id': 'req-cabd5a2d-a91a-4839-8479-6b8d731f0161'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:28.129649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] The token cache is being invalidate because project 8bf2245b0b9147bfb06a9ab8dad145af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:28.130096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:28.130771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:28.133057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 975/1949] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:27 2022] DELETE /identity/v3/projects/8bf2245b0b9147bfb06a9ab8dad145af => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:28.136836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:28.142674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:28.188913 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f8e5f-b98d-4c61-a594-3c7029c207f1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:28.196177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56ee70-9194-44fa-a1a3-4e1ce6741126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:28.196806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56ee70-9194-44fa-a1a3-4e1ce6741126 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.197158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56ee70-9194-44fa-a1a3-4e1ce6741126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.197503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f56ee70-9194-44fa-a1a3-4e1ce6741126 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.198209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56ee70-9194-44fa-a1a3-4e1ce6741126 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.200853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f56ee70-9194-44fa-a1a3-4e1ce6741126 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.229261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 975/1950] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:28 2022] POST /identity/v3/users => generated 296 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:28.235459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56ee70-9194-44fa-a1a3-4e1ce6741126 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:28.237697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabd5a2d-a91a-4839-8479-6b8d731f0161 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:28.238434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7f56ee70-9194-44fa-a1a3-4e1ce6741126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:28.242549 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:28.243327 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.243797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.244073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] PATH_INFO: `/v3/users/051e339881004ce7821a513bb4b11798` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.245389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] RBAC: Authorizing `identity:delete_user(user_id=051e339881004ce7821a513bb4b11798)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.248535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.254723 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f56ee70-9194-44fa-a1a3-4e1ce6741126 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:28.264552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a6d8-c172-4e6b-aa2f-a609810d6d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:28.265221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a6d8-c172-4e6b-aa2f-a609810d6d3e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.265497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a6d8-c172-4e6b-aa2f-a609810d6d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.265761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538a6d8-c172-4e6b-aa2f-a609810d6d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.266630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538a6d8-c172-4e6b-aa2f-a609810d6d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.268330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538a6d8-c172-4e6b-aa2f-a609810d6d3e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.284981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 976/1951] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:28.286019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a6d8-c172-4e6b-aa2f-a609810d6d3e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:28.298363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538a6d8-c172-4e6b-aa2f-a609810d6d3e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:28.303586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21901d-cbf9-47e5-9309-fa6036d11606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:28.304241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21901d-cbf9-47e5-9309-fa6036d11606 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.304531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21901d-cbf9-47e5-9309-fa6036d11606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.304805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21901d-cbf9-47e5-9309-fa6036d11606 admin admin] PATH_INFO: `/v3/projects/96f8be7d7587438a94f503c513822fe6/users/69282d7ac6cc472dad71acb1cac27a3a/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.344292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '051e339881004ce7821a513bb4b11798'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:28.345054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '051e339881004ce7821a513bb4b11798'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:28.348351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21901d-cbf9-47e5-9309-fa6036d11606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f8be7d7587438a94f503c513822fe6, user_id=69282d7ac6cc472dad71acb1cac27a3a, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.352464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21901d-cbf9-47e5-9309-fa6036d11606 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.355991 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '051e339881004ce7821a513bb4b11798'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:28.365556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '051e339881004ce7821a513bb4b11798'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:28.366253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '051e339881004ce7821a513bb4b11798'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:28.368453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 977/1952] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:28 2022] PUT /identity/v3/projects/96f8be7d7587438a94f503c513822fe6/users/69282d7ac6cc472dad71acb1cac27a3a/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:28.373140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21901d-cbf9-47e5-9309-fa6036d11606 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:28.375080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4a21901d-cbf9-47e5-9309-fa6036d11606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:28.382116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:28.393326 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 976/1953] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:28 2022] DELETE /identity/v3/users/051e339881004ce7821a513bb4b11798 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:28.452555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21901d-cbf9-47e5-9309-fa6036d11606 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:28.526724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dab866-7457-4894-9f97-576b135a3d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:28.527356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dab866-7457-4894-9f97-576b135a3d0e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.528322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dab866-7457-4894-9f97-576b135a3d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.528807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dab866-7457-4894-9f97-576b135a3d0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.529699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dab866-7457-4894-9f97-576b135a3d0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.531366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dab866-7457-4894-9f97-576b135a3d0e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.541669 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 978/1954] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:28 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:28.546167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:28.559388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436d4023-51c0-48df-80db-4bc3bfdf2421 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:28.568175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d503b95-0a04-4361-b0b4-926ad5bfa1c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:28.569076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d503b95-0a04-4361-b0b4-926ad5bfa1c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.569735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d503b95-0a04-4361-b0b4-926ad5bfa1c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.570114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d503b95-0a04-4361-b0b4-926ad5bfa1c2 admin admin] PATH_INFO: `/v3/projects/96f8be7d7587438a94f503c513822fe6/users/69282d7ac6cc472dad71acb1cac27a3a/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.572591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d503b95-0a04-4361-b0b4-926ad5bfa1c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96f8be7d7587438a94f503c513822fe6, user_id=69282d7ac6cc472dad71acb1cac27a3a, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.576777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d503b95-0a04-4361-b0b4-926ad5bfa1c2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.590034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 977/1955] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:28 2022] PUT /identity/v3/projects/96f8be7d7587438a94f503c513822fe6/users/69282d7ac6cc472dad71acb1cac27a3a/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:28.601133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8d49-01b2-431a-8c6b-928a0d459605 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.601466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8d49-01b2-431a-8c6b-928a0d459605 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.601774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fde8d49-01b2-431a-8c6b-928a0d459605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.624536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:28.640209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0fde8d49-01b2-431a-8c6b-928a0d459605 None None] MFA Rules not processed for user `69282d7ac6cc472dad71acb1cac27a3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:28.642310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0fde8d49-01b2-431a-8c6b-928a0d459605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:28.722416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d503b95-0a04-4361-b0b4-926ad5bfa1c2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:28.724628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9d503b95-0a04-4361-b0b4-926ad5bfa1c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:28.792394 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 979/1956] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:28 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:28.807592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde8d49-01b2-431a-8c6b-928a0d459605 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:28.810725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0fde8d49-01b2-431a-8c6b-928a0d459605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:28.877873 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d503b95-0a04-4361-b0b4-926ad5bfa1c2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:28.886719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:28.887994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.888556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.888794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] PATH_INFO: `/v3/projects/887c0b2bf0e54ee18c28ee1e90430efd` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.890028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] RBAC: Authorizing `identity:delete_project(project_id=887c0b2bf0e54ee18c28ee1e90430efd)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.891744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fde8d49-01b2-431a-8c6b-928a0d459605 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:28.892659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.904792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf2ce6-3b43-414d-8fe0-dad4904f1eb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:28.905594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf2ce6-3b43-414d-8fe0-dad4904f1eb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:28.906574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf2ce6-3b43-414d-8fe0-dad4904f1eb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:28.907063 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf2ce6-3b43-414d-8fe0-dad4904f1eb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:28.919084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf2ce6-3b43-414d-8fe0-dad4904f1eb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:28.920867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf2ce6-3b43-414d-8fe0-dad4904f1eb5 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:28.946725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 980/1957] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:29.002496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '887c0b2bf0e54ee18c28ee1e90430efd', 'request_id': 'req-86b5aa09-f036-45cb-be13-8518b06b549a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:29.020093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '887c0b2bf0e54ee18c28ee1e90430efd', 'request_id': 'req-86b5aa09-f036-45cb-be13-8518b06b549a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:29.026619 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '887c0b2bf0e54ee18c28ee1e90430efd', 'request_id': 'req-86b5aa09-f036-45cb-be13-8518b06b549a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:29.052118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] The token cache is being invalidate because project 887c0b2bf0e54ee18c28ee1e90430efd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:29.052409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:29.053074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-86b5aa09-f036-45cb-be13-8518b06b549a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:29.058864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 978/1958] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:28 2022] DELETE /identity/v3/projects/887c0b2bf0e54ee18c28ee1e90430efd => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:29.060931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb83e1-50f2-4a77-9e6e-bed705c2388b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.061264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb83e1-50f2-4a77-9e6e-bed705c2388b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.061461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb83e1-50f2-4a77-9e6e-bed705c2388b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.086813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-32cb83e1-50f2-4a77-9e6e-bed705c2388b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:29.088220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-32cb83e1-50f2-4a77-9e6e-bed705c2388b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:29.194181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 981/1959] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:29.202808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3f70a-2a73-4279-8d53-de047209ba40 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.203321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3f70a-2a73-4279-8d53-de047209ba40 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.203321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda3f70a-2a73-4279-8d53-de047209ba40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.228745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-dda3f70a-2a73-4279-8d53-de047209ba40 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:29.228745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-dda3f70a-2a73-4279-8d53-de047209ba40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:29.244188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 979/1960] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:29.249733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb83e1-50f2-4a77-9e6e-bed705c2388b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:29.260227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb83e1-50f2-4a77-9e6e-bed705c2388b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:29.266973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81865bbf-3b7c-4466-8f73-b9e65c3f7d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:29.267622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81865bbf-3b7c-4466-8f73-b9e65c3f7d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.267934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81865bbf-3b7c-4466-8f73-b9e65c3f7d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.268208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81865bbf-3b7c-4466-8f73-b9e65c3f7d87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.268869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81865bbf-3b7c-4466-8f73-b9e65c3f7d87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:29.269174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81865bbf-3b7c-4466-8f73-b9e65c3f7d87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:29.270786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81865bbf-3b7c-4466-8f73-b9e65c3f7d87 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:29.291658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 982/1961] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:29.292580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3f70a-2a73-4279-8d53-de047209ba40 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:29.304457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda3f70a-2a73-4279-8d53-de047209ba40 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:29.310224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365a0be-3f57-46b5-b6d4-89f8a119a4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:29.310917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365a0be-3f57-46b5-b6d4-89f8a119a4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.311259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365a0be-3f57-46b5-b6d4-89f8a119a4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.311573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4365a0be-3f57-46b5-b6d4-89f8a119a4a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.312215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365a0be-3f57-46b5-b6d4-89f8a119a4a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:29.314356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4365a0be-3f57-46b5-b6d4-89f8a119a4a3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:29.356099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 980/1962] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:29 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:29.360524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81865bbf-3b7c-4466-8f73-b9e65c3f7d87 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:29.372244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81865bbf-3b7c-4466-8f73-b9e65c3f7d87 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:29.401341 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:29.438170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365a0be-3f57-46b5-b6d4-89f8a119a4a3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:29.440016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4365a0be-3f57-46b5-b6d4-89f8a119a4a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:29.482169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7b1bf-af84-4c4b-abe9-678402da3753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:29.482806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7b1bf-af84-4c4b-abe9-678402da3753 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.483071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7b1bf-af84-4c4b-abe9-678402da3753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.483343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a7b1bf-af84-4c4b-abe9-678402da3753 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.484017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7b1bf-af84-4c4b-abe9-678402da3753 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:29.486237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a7b1bf-af84-4c4b-abe9-678402da3753 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:29.517319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 983/1963] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:29 2022] POST /identity/v3/users => generated 287 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:29.522189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:29.522734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7b1bf-af84-4c4b-abe9-678402da3753 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:29.544172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a7b1bf-af84-4c4b-abe9-678402da3753 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:29.549581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5293008-bb5e-4b8e-afe4-527b43c6e5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:29.550345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5293008-bb5e-4b8e-afe4-527b43c6e5ef admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.550689 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5293008-bb5e-4b8e-afe4-527b43c6e5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.551130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5293008-bb5e-4b8e-afe4-527b43c6e5ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.551971 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5293008-bb5e-4b8e-afe4-527b43c6e5ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:29.554650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5293008-bb5e-4b8e-afe4-527b43c6e5ef admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:29.558306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4365a0be-3f57-46b5-b6d4-89f8a119a4a3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:29.565519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:29.566530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.567112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.567470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] PATH_INFO: `/v3/users/9dda839a764b4fdfb64d267ac383bf18` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.569681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dda839a764b4fdfb64d267ac383bf18)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:29.573759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 984/1964] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:29.575248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:29.576795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5293008-bb5e-4b8e-afe4-527b43c6e5ef admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:29.600634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5293008-bb5e-4b8e-afe4-527b43c6e5ef admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:29.608478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2b945-ba35-4c2b-bf87-d1dd97d7b044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:29.609541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2b945-ba35-4c2b-bf87-d1dd97d7b044 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.610695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2b945-ba35-4c2b-bf87-d1dd97d7b044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.611142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2b945-ba35-4c2b-bf87-d1dd97d7b044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.612152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2b945-ba35-4c2b-bf87-d1dd97d7b044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:29.613950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2b945-ba35-4c2b-bf87-d1dd97d7b044 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:29.632461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 985/1965] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:29 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:29.635698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2b945-ba35-4c2b-bf87-d1dd97d7b044 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:29.642181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dda839a764b4fdfb64d267ac383bf18'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:29.642880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dda839a764b4fdfb64d267ac383bf18'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:29.650903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dda839a764b4fdfb64d267ac383bf18'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:29.657093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2b945-ba35-4c2b-bf87-d1dd97d7b044 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:29.662832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f50b8b-d555-4f24-9aad-1b214116053c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:29.663993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f50b8b-d555-4f24-9aad-1b214116053c admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.664415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f50b8b-d555-4f24-9aad-1b214116053c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.665333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f50b8b-d555-4f24-9aad-1b214116053c admin admin] PATH_INFO: `/v3/projects/1e21730dd5cf4ebd8edf738ff0b02cc5/users/b5ac74507a2c4f82a0508145d9ea89dd/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.670482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dda839a764b4fdfb64d267ac383bf18'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:29.672816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-98b5a408-8ddb-4943-9d9a-0069ce958c82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dda839a764b4fdfb64d267ac383bf18'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:29.690679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f50b8b-d555-4f24-9aad-1b214116053c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e21730dd5cf4ebd8edf738ff0b02cc5, user_id=b5ac74507a2c4f82a0508145d9ea89dd, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:29.693578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f50b8b-d555-4f24-9aad-1b214116053c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:29.724574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 981/1966] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:29 2022] DELETE /identity/v3/users/9dda839a764b4fdfb64d267ac383bf18 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:29.728954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640ce40-bd4b-43a6-ac41-63523d9e1fbc None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.729236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640ce40-bd4b-43a6-ac41-63523d9e1fbc None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.729558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640ce40-bd4b-43a6-ac41-63523d9e1fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.740220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 986/1967] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:29 2022] PUT /identity/v3/projects/1e21730dd5cf4ebd8edf738ff0b02cc5/users/b5ac74507a2c4f82a0508145d9ea89dd/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:29.742088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2cb81-45df-4f1f-8c0a-af57bd46173f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.742398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2cb81-45df-4f1f-8c0a-af57bd46173f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.742672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2cb81-45df-4f1f-8c0a-af57bd46173f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.767407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7640ce40-bd4b-43a6-ac41-63523d9e1fbc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:29.769028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7640ce40-bd4b-43a6-ac41-63523d9e1fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:29.796363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7ed2cb81-45df-4f1f-8c0a-af57bd46173f None None] MFA Rules not processed for user `b5ac74507a2c4f82a0508145d9ea89dd`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:29.797661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7ed2cb81-45df-4f1f-8c0a-af57bd46173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:29.878698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 987/1968] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:29.885581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 982/1969] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:29.893530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2cb81-45df-4f1f-8c0a-af57bd46173f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:29.894909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7ed2cb81-45df-4f1f-8c0a-af57bd46173f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:29.896554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640ce40-bd4b-43a6-ac41-63523d9e1fbc None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:29.898063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7640ce40-bd4b-43a6-ac41-63523d9e1fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:29.954045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2cb81-45df-4f1f-8c0a-af57bd46173f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:29.957727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb68ea7-4ba4-4268-8ec1-0cbb77fffe65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:29.958339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb68ea7-4ba4-4268-8ec1-0cbb77fffe65 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.958616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb68ea7-4ba4-4268-8ec1-0cbb77fffe65 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.958890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb68ea7-4ba4-4268-8ec1-0cbb77fffe65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.959299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640ce40-bd4b-43a6-ac41-63523d9e1fbc None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:29.963745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbba7ee0-6c90-441c-8069-5d7a5b22c087 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:29.964358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba7ee0-6c90-441c-8069-5d7a5b22c087 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:29.964633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba7ee0-6c90-441c-8069-5d7a5b22c087 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:29.964906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbba7ee0-6c90-441c-8069-5d7a5b22c087 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:29.969480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb68ea7-4ba4-4268-8ec1-0cbb77fffe65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:29.971186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb68ea7-4ba4-4268-8ec1-0cbb77fffe65 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:29.976215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba7ee0-6c90-441c-8069-5d7a5b22c087 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:29.976737 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:29.977648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbba7ee0-6c90-441c-8069-5d7a5b22c087 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:29.983431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 988/1970] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:29.997059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 983/1971] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:30.019126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb68ea7-4ba4-4268-8ec1-0cbb77fffe65 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:30.021416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2fb68ea7-4ba4-4268-8ec1-0cbb77fffe65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:30.037193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb68ea7-4ba4-4268-8ec1-0cbb77fffe65 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:30.045192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:30.045838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:30.047490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:30.047490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] PATH_INFO: `/v3/users/b5ac74507a2c4f82a0508145d9ea89dd` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:30.049645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ac74507a2c4f82a0508145d9ea89dd)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:30.052097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:30.125023 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ac74507a2c4f82a0508145d9ea89dd'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:30.134151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ac74507a2c4f82a0508145d9ea89dd'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:30.134857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ac74507a2c4f82a0508145d9ea89dd'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:30.135697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ac74507a2c4f82a0508145d9ea89dd'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:30.145566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ac74507a2c4f82a0508145d9ea89dd'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:30.200722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 989/1972] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:30 2022] DELETE /identity/v3/users/b5ac74507a2c4f82a0508145d9ea89dd => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:30.201691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b20bd-e18f-41ea-a552-a770049abe0b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:30.201981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b20bd-e18f-41ea-a552-a770049abe0b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:30.202259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4b20bd-e18f-41ea-a552-a770049abe0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:30.217699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:30.236314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7a4b20bd-e18f-41ea-a552-a770049abe0b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:30.238711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7a4b20bd-e18f-41ea-a552-a770049abe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:30.316526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 984/1973] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:30.337060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:30.339481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:30.417369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356265a-cb4d-4520-bc87-22ba510409f4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:30.421902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81046d49-2aa6-496f-a470-57de57345aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:30.422983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046d49-2aa6-496f-a470-57de57345aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:30.423261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046d49-2aa6-496f-a470-57de57345aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:30.423564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81046d49-2aa6-496f-a470-57de57345aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:30.433508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81046d49-2aa6-496f-a470-57de57345aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:30.435030 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81046d49-2aa6-496f-a470-57de57345aa7 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:30.446172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 990/1974] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:30.964898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b20bd-e18f-41ea-a552-a770049abe0b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:30.967312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7a4b20bd-e18f-41ea-a552-a770049abe0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:30.988260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4b20bd-e18f-41ea-a552-a770049abe0b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:30.994651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:30.995881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:30.997045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:30.998584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] PATH_INFO: `/v3/projects/7fedadec09044f8a9bd7277fb57e7276` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.001088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fedadec09044f8a9bd7277fb57e7276)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:31.003846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81046d49-2aa6-496f-a470-57de57345aa7 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:31.006267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-81046d49-2aa6-496f-a470-57de57345aa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:31.016025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:31.035019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81046d49-2aa6-496f-a470-57de57345aa7 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:31.056851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:31.057501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.057777 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.058042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] PATH_INFO: `/v3/projects/1e21730dd5cf4ebd8edf738ff0b02cc5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.060763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e21730dd5cf4ebd8edf738ff0b02cc5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:31.062748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:31.125845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fedadec09044f8a9bd7277fb57e7276', 'request_id': 'req-d544881c-5ef7-40a4-8cda-18e890589187'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.137950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fedadec09044f8a9bd7277fb57e7276', 'request_id': 'req-d544881c-5ef7-40a4-8cda-18e890589187'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.148336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fedadec09044f8a9bd7277fb57e7276', 'request_id': 'req-d544881c-5ef7-40a4-8cda-18e890589187'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.172161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e21730dd5cf4ebd8edf738ff0b02cc5', 'request_id': 'req-c74bacb6-474d-46c2-bc34-0548c498ec86'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.192331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] The token cache is being invalidate because project 7fedadec09044f8a9bd7277fb57e7276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:31.192933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.195028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.197646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e21730dd5cf4ebd8edf738ff0b02cc5', 'request_id': 'req-c74bacb6-474d-46c2-bc34-0548c498ec86'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.197776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 985/1975] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:30 2022] DELETE /identity/v3/projects/7fedadec09044f8a9bd7277fb57e7276 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:31.205051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:31.207750 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:31.217784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e21730dd5cf4ebd8edf738ff0b02cc5', 'request_id': 'req-c74bacb6-474d-46c2-bc34-0548c498ec86'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.233105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:31.244137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] The token cache is being invalidate because project 1e21730dd5cf4ebd8edf738ff0b02cc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:31.245046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.246773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c74bacb6-474d-46c2-bc34-0548c498ec86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.255333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 991/1976] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:31 2022] DELETE /identity/v3/projects/1e21730dd5cf4ebd8edf738ff0b02cc5 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:31.258812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632be64e-8700-4e97-a4f2-7c4b3728272a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.259426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632be64e-8700-4e97-a4f2-7c4b3728272a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.259774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632be64e-8700-4e97-a4f2-7c4b3728272a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.313189 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-632be64e-8700-4e97-a4f2-7c4b3728272a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:31.314563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-632be64e-8700-4e97-a4f2-7c4b3728272a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:31.332619 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d544881c-5ef7-40a4-8cda-18e890589187 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:31.379356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:31.417467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:31.418771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.419245 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.419719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] PATH_INFO: `/v3/users/69282d7ac6cc472dad71acb1cac27a3a` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.428995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69282d7ac6cc472dad71acb1cac27a3a)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:31.428995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:31.439505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 992/1977] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:31.466487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18d98b-2e64-4f38-be37-bc87821738de None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.466917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18d98b-2e64-4f38-be37-bc87821738de None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.467358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f18d98b-2e64-4f38-be37-bc87821738de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.494958 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4f18d98b-2e64-4f38-be37-bc87821738de None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:31.497626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4f18d98b-2e64-4f38-be37-bc87821738de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:31.513660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 993/1978] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:31.514621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18d98b-2e64-4f38-be37-bc87821738de None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:31.525471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f18d98b-2e64-4f38-be37-bc87821738de None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:31.536305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69282d7ac6cc472dad71acb1cac27a3a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.536959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69282d7ac6cc472dad71acb1cac27a3a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.546056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d26649-cedb-45c8-ae39-c82f7b9a240f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:31.546685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26649-cedb-45c8-ae39-c82f7b9a240f admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.546955 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26649-cedb-45c8-ae39-c82f7b9a240f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.547224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d26649-cedb-45c8-ae39-c82f7b9a240f admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.547969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69282d7ac6cc472dad71acb1cac27a3a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.548295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d26649-cedb-45c8-ae39-c82f7b9a240f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:31.548505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d26649-cedb-45c8-ae39-c82f7b9a240f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:31.549945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d26649-cedb-45c8-ae39-c82f7b9a240f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:31.558374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69282d7ac6cc472dad71acb1cac27a3a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.560662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69282d7ac6cc472dad71acb1cac27a3a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:31.563412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 994/1979] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:31.567661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d26649-cedb-45c8-ae39-c82f7b9a240f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:31.570290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-29d26649-cedb-45c8-ae39-c82f7b9a240f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:31.584475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d26649-cedb-45c8-ae39-c82f7b9a240f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:31.585948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 986/1980] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:31 2022] DELETE /identity/v3/users/69282d7ac6cc472dad71acb1cac27a3a => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:31.664703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d3e6b-16f3-4c67-9bf2-0dc1a32d5f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:31.665352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3e6b-16f3-4c67-9bf2-0dc1a32d5f79 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.665722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3e6b-16f3-4c67-9bf2-0dc1a32d5f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.666022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d3e6b-16f3-4c67-9bf2-0dc1a32d5f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.666783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d3e6b-16f3-4c67-9bf2-0dc1a32d5f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:31.669622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d3e6b-16f3-4c67-9bf2-0dc1a32d5f79 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:31.707573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 995/1981] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:31 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:31.712308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:31.725617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d3cbd3-ab4f-4d76-b98b-ea213b93a5f4 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:31.798815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e3833-5590-4fd1-8b6b-57fec878c8dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:31.799459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e3833-5590-4fd1-8b6b-57fec878c8dd admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.799748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e3833-5590-4fd1-8b6b-57fec878c8dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.800021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e3833-5590-4fd1-8b6b-57fec878c8dd admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.800663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e3833-5590-4fd1-8b6b-57fec878c8dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:31.803896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e3833-5590-4fd1-8b6b-57fec878c8dd admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:31.835892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 987/1982] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:31 2022] POST /identity/v3/users => generated 292 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:31.836597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d3e6b-16f3-4c67-9bf2-0dc1a32d5f79 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:31.862335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d3e6b-16f3-4c67-9bf2-0dc1a32d5f79 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:31.869045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc499c-6489-4bb9-b136-6b845f796585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:31.869717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc499c-6489-4bb9-b136-6b845f796585 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.870062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc499c-6489-4bb9-b136-6b845f796585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.870376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc499c-6489-4bb9-b136-6b845f796585 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.871045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc499c-6489-4bb9-b136-6b845f796585 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:31.872983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc499c-6489-4bb9-b136-6b845f796585 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:31.884212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 996/1983] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:31.888730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e3833-5590-4fd1-8b6b-57fec878c8dd admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:31.899174 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e3833-5590-4fd1-8b6b-57fec878c8dd admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:31.904731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7a0f7-5981-42de-87d3-9237d7d0877f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:31.905353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7a0f7-5981-42de-87d3-9237d7d0877f admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.905632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7a0f7-5981-42de-87d3-9237d7d0877f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.905905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7a0f7-5981-42de-87d3-9237d7d0877f admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.906558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7a0f7-5981-42de-87d3-9237d7d0877f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:31.908239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7a0f7-5981-42de-87d3-9237d7d0877f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:31.920046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 988/1984] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:31.924242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc499c-6489-4bb9-b136-6b845f796585 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:31.935047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc499c-6489-4bb9-b136-6b845f796585 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:31.940061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cbb2e-106a-4816-b0ef-f2822a708084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:31.940814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cbb2e-106a-4816-b0ef-f2822a708084 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.941170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cbb2e-106a-4816-b0ef-f2822a708084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.941528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42cbb2e-106a-4816-b0ef-f2822a708084 admin admin] PATH_INFO: `/v3/projects/e3a9c95bc33f4697917a9e8c53f78a98/users/8f468d0b1fec4502858b8e29ac640ebd/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:31.957598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cbb2e-106a-4816-b0ef-f2822a708084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3a9c95bc33f4697917a9e8c53f78a98, user_id=8f468d0b1fec4502858b8e29ac640ebd, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:31.960640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42cbb2e-106a-4816-b0ef-f2822a708084 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:31.975004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 997/1985] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:31 2022] PUT /identity/v3/projects/e3a9c95bc33f4697917a9e8c53f78a98/users/8f468d0b1fec4502858b8e29ac640ebd/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:31.980612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabc299-dd84-4b7d-a076-7a06e2ef89de None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:31.980996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabc299-dd84-4b7d-a076-7a06e2ef89de None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:31.981332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabc299-dd84-4b7d-a076-7a06e2ef89de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.004256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-0fabc299-dd84-4b7d-a076-7a06e2ef89de None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:32.005567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0fabc299-dd84-4b7d-a076-7a06e2ef89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:32.036633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:32.071819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 989/1986] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:32.082321 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cbb2e-106a-4816-b0ef-f2822a708084 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:32.084723 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e42cbb2e-106a-4816-b0ef-f2822a708084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:32.122941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:32.128249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabc299-dd84-4b7d-a076-7a06e2ef89de None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:32.129663 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0fabc299-dd84-4b7d-a076-7a06e2ef89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:32.144977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabc299-dd84-4b7d-a076-7a06e2ef89de None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:32.156274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42cbb2e-106a-4816-b0ef-f2822a708084 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:32.160206 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59e015-bae9-4950-bbab-9d942598784c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:32.160817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59e015-bae9-4950-bbab-9d942598784c service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.161093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59e015-bae9-4950-bbab-9d942598784c service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.161385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59e015-bae9-4950-bbab-9d942598784c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.167298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:32.168842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.169129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.171381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] PATH_INFO: `/v3/projects/96f8be7d7587438a94f503c513822fe6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.172019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59e015-bae9-4950-bbab-9d942598784c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:32.173473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59e015-bae9-4950-bbab-9d942598784c service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:32.174374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96f8be7d7587438a94f503c513822fe6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:32.190793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:32.191808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 998/1987] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:32.235246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96f8be7d7587438a94f503c513822fe6', 'request_id': 'req-f1e75b65-3273-4bfb-aa3a-de34121b3a90'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:32.243840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96f8be7d7587438a94f503c513822fe6', 'request_id': 'req-f1e75b65-3273-4bfb-aa3a-de34121b3a90'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:32.250093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96f8be7d7587438a94f503c513822fe6', 'request_id': 'req-f1e75b65-3273-4bfb-aa3a-de34121b3a90'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:32.261606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] The token cache is being invalidate because project 96f8be7d7587438a94f503c513822fe6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:32.262436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:32.263112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f1e75b65-3273-4bfb-aa3a-de34121b3a90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:32.264783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 990/1988] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:32 2022] DELETE /identity/v3/projects/96f8be7d7587438a94f503c513822fe6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:32.270047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff2161-d435-4633-9390-f0dd4be5b495 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.270333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff2161-d435-4633-9390-f0dd4be5b495 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.270609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aff2161-d435-4633-9390-f0dd4be5b495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.292376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8aff2161-d435-4633-9390-f0dd4be5b495 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:32.293636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8aff2161-d435-4633-9390-f0dd4be5b495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:32.365617 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 999/1989] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:32.367475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998db4f-7c4c-4d26-9324-b0d93b7eec4b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.367928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998db4f-7c4c-4d26-9324-b0d93b7eec4b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.368030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c998db4f-7c4c-4d26-9324-b0d93b7eec4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.391044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c998db4f-7c4c-4d26-9324-b0d93b7eec4b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:32.392413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c998db4f-7c4c-4d26-9324-b0d93b7eec4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:32.403591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 991/1990] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:32.412253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff2161-d435-4633-9390-f0dd4be5b495 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:32.426355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aff2161-d435-4633-9390-f0dd4be5b495 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:32.434650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f4823-2a0f-4517-9d5a-086924de7d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:32.435549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4823-2a0f-4517-9d5a-086924de7d24 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.435901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4823-2a0f-4517-9d5a-086924de7d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.436266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474f4823-2a0f-4517-9d5a-086924de7d24 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.437061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f4823-2a0f-4517-9d5a-086924de7d24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:32.437429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f4823-2a0f-4517-9d5a-086924de7d24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:32.439211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474f4823-2a0f-4517-9d5a-086924de7d24 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:32.453999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1000/1991] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:32.458345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998db4f-7c4c-4d26-9324-b0d93b7eec4b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:32.469087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c998db4f-7c4c-4d26-9324-b0d93b7eec4b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:32.491784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441bcaa1-d519-4071-90f5-39d14df2006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:32.492387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bcaa1-d519-4071-90f5-39d14df2006e admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.492740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bcaa1-d519-4071-90f5-39d14df2006e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.493091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441bcaa1-d519-4071-90f5-39d14df2006e admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.493786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441bcaa1-d519-4071-90f5-39d14df2006e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:32.495975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441bcaa1-d519-4071-90f5-39d14df2006e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:32.534284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 992/1992] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:32 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:32.535922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f4823-2a0f-4517-9d5a-086924de7d24 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:32.551134 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474f4823-2a0f-4517-9d5a-086924de7d24 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:32.638686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b100220-394b-4c3a-ad48-35c12eaf922b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:32.639378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b100220-394b-4c3a-ad48-35c12eaf922b admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.639755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b100220-394b-4c3a-ad48-35c12eaf922b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.640161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b100220-394b-4c3a-ad48-35c12eaf922b admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.640988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b100220-394b-4c3a-ad48-35c12eaf922b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:32.644554 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b100220-394b-4c3a-ad48-35c12eaf922b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:32.670282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1001/1993] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:32 2022] POST /identity/v3/users => generated 292 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:32.674672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441bcaa1-d519-4071-90f5-39d14df2006e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:32.688595 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441bcaa1-d519-4071-90f5-39d14df2006e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:32.695012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9231ab-6195-43c7-93c4-80800d18a416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:32.695896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9231ab-6195-43c7-93c4-80800d18a416 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.696362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9231ab-6195-43c7-93c4-80800d18a416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.696815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9231ab-6195-43c7-93c4-80800d18a416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.697667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9231ab-6195-43c7-93c4-80800d18a416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:32.700409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9231ab-6195-43c7-93c4-80800d18a416 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:32.717939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b100220-394b-4c3a-ad48-35c12eaf922b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:32.728136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 993/1994] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:32.728677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b100220-394b-4c3a-ad48-35c12eaf922b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:32.735081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d4015-4e53-47fe-a59a-03c54076b784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:32.735899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d4015-4e53-47fe-a59a-03c54076b784 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.736190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d4015-4e53-47fe-a59a-03c54076b784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.736467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d4015-4e53-47fe-a59a-03c54076b784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.737112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d4015-4e53-47fe-a59a-03c54076b784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:32.739168 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d4015-4e53-47fe-a59a-03c54076b784 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:32.801869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1002/1995] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:32.806280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9231ab-6195-43c7-93c4-80800d18a416 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:32.819387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9231ab-6195-43c7-93c4-80800d18a416 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:32.827108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5e24d-7246-49c3-8c9b-92d4f46ad6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:32.827943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5e24d-7246-49c3-8c9b-92d4f46ad6c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.828347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5e24d-7246-49c3-8c9b-92d4f46ad6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.828853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5e24d-7246-49c3-8c9b-92d4f46ad6c9 admin admin] PATH_INFO: `/v3/projects/3cfd25960fb64bd0a9d45111acc35f17/users/0aa1640df3374167ae736cffc0aab3aa/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.852787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5e24d-7246-49c3-8c9b-92d4f46ad6c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cfd25960fb64bd0a9d45111acc35f17, user_id=0aa1640df3374167ae736cffc0aab3aa, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:32.867350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5e24d-7246-49c3-8c9b-92d4f46ad6c9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:32.902353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 994/1996] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:32 2022] PUT /identity/v3/projects/3cfd25960fb64bd0a9d45111acc35f17/users/0aa1640df3374167ae736cffc0aab3aa/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:32.904934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa990837-0b9f-4c48-9b3c-e5a327d74443 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:32.905441 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa990837-0b9f-4c48-9b3c-e5a327d74443 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:32.905898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa990837-0b9f-4c48-9b3c-e5a327d74443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:32.942407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-aa990837-0b9f-4c48-9b3c-e5a327d74443 None None] MFA Rules not processed for user `0aa1640df3374167ae736cffc0aab3aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:32.944023 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-aa990837-0b9f-4c48-9b3c-e5a327d74443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:32.974400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:33.046254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1003/1997] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:33.059997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5e24d-7246-49c3-8c9b-92d4f46ad6c9 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:33.062207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4bb5e24d-7246-49c3-8c9b-92d4f46ad6c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:33.137457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5e24d-7246-49c3-8c9b-92d4f46ad6c9 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:33.141780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eaa00-46ff-44a4-b177-a3ee9063652c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:33.142199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eaa00-46ff-44a4-b177-a3ee9063652c service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:33.142623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eaa00-46ff-44a4-b177-a3ee9063652c service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:33.142904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0eaa00-46ff-44a4-b177-a3ee9063652c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:33.151279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:33.154043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eaa00-46ff-44a4-b177-a3ee9063652c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:33.156209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0eaa00-46ff-44a4-b177-a3ee9063652c service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:33.173265 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 995/1998] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:33.285710 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d1bd8-6b3f-447b-9408-d8466a6ce864 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:33.286181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d1bd8-6b3f-447b-9408-d8466a6ce864 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:33.286468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1d1bd8-6b3f-447b-9408-d8466a6ce864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:33.350145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bb1d1bd8-6b3f-447b-9408-d8466a6ce864 None None] MFA Rules not processed for user `8f468d0b1fec4502858b8e29ac640ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:33.352573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bb1d1bd8-6b3f-447b-9408-d8466a6ce864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:33.486393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1004/1999] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:33.499289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eaa00-46ff-44a4-b177-a3ee9063652c service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:33.504414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ca0eaa00-46ff-44a4-b177-a3ee9063652c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:33.521472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0eaa00-46ff-44a4-b177-a3ee9063652c service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:33.536293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341d24c-e3ba-4b0a-8a8e-396757447100 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:33.536993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341d24c-e3ba-4b0a-8a8e-396757447100 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:33.537776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341d24c-e3ba-4b0a-8a8e-396757447100 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:33.538140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341d24c-e3ba-4b0a-8a8e-396757447100 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:33.561865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341d24c-e3ba-4b0a-8a8e-396757447100 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:33.565946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341d24c-e3ba-4b0a-8a8e-396757447100 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:33.578818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 996/2000] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:36.088924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde1609-3d0e-473d-ac17-a890d01ba639 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:36.089546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde1609-3d0e-473d-ac17-a890d01ba639 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:36.089546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cde1609-3d0e-473d-ac17-a890d01ba639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:36.110060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1cde1609-3d0e-473d-ac17-a890d01ba639 None None] MFA Rules not processed for user `8f468d0b1fec4502858b8e29ac640ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:36.111553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1cde1609-3d0e-473d-ac17-a890d01ba639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:36.121823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1005/2001] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:43.245960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341d24c-e3ba-4b0a-8a8e-396757447100 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:43.338363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341d24c-e3ba-4b0a-8a8e-396757447100 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:43.345361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:43.346011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:43.346305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:43.346574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] PATH_INFO: `/v3/users/0aa1640df3374167ae736cffc0aab3aa` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:43.347811 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0aa1640df3374167ae736cffc0aab3aa)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:43.350531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:43.411049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0aa1640df3374167ae736cffc0aab3aa'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:43.411718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0aa1640df3374167ae736cffc0aab3aa'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:43.420476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0aa1640df3374167ae736cffc0aab3aa'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:43.431023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0aa1640df3374167ae736cffc0aab3aa'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:43.431782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0aa1640df3374167ae736cffc0aab3aa'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:43.456751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 997/2002] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:43 2022] DELETE /identity/v3/users/0aa1640df3374167ae736cffc0aab3aa => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:43.462340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5bf98-3162-4cd8-accf-c5c95a06f536 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:43.462847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5bf98-3162-4cd8-accf-c5c95a06f536 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:43.462937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5bf98-3162-4cd8-accf-c5c95a06f536 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:43.483494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-72a5bf98-3162-4cd8-accf-c5c95a06f536 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:43.484735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-72a5bf98-3162-4cd8-accf-c5c95a06f536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:43.564297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1006/2003] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:43.574009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:43.575952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:43.677674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67d12ca-c3fa-47b7-b535-9839c4afc0cb admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:43.694713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2267b841-8b1a-4ff5-975b-9f1350967106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:43.695584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267b841-8b1a-4ff5-975b-9f1350967106 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:43.695962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267b841-8b1a-4ff5-975b-9f1350967106 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:43.696329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2267b841-8b1a-4ff5-975b-9f1350967106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:43.714191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267b841-8b1a-4ff5-975b-9f1350967106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:43.716458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2267b841-8b1a-4ff5-975b-9f1350967106 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:43.727202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 998/2004] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:44.461795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5bf98-3162-4cd8-accf-c5c95a06f536 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:44.463475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-72a5bf98-3162-4cd8-accf-c5c95a06f536 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:44.476923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5bf98-3162-4cd8-accf-c5c95a06f536 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:44.483563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:44.484283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:44.484587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:44.484967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] PATH_INFO: `/v3/projects/3cfd25960fb64bd0a9d45111acc35f17` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:44.486653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cfd25960fb64bd0a9d45111acc35f17)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:44.488634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:44.573125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:44.577309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cfd25960fb64bd0a9d45111acc35f17', 'request_id': 'req-885743d9-4005-46c8-b6bb-19d3bd9b9be4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:44.584109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cfd25960fb64bd0a9d45111acc35f17', 'request_id': 'req-885743d9-4005-46c8-b6bb-19d3bd9b9be4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:44.592349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cfd25960fb64bd0a9d45111acc35f17', 'request_id': 'req-885743d9-4005-46c8-b6bb-19d3bd9b9be4'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:44.605779 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] The token cache is being invalidate because project 3cfd25960fb64bd0a9d45111acc35f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:44.606080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:44.607028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-885743d9-4005-46c8-b6bb-19d3bd9b9be4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:44.608861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1007/2005] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:44 2022] DELETE /identity/v3/projects/3cfd25960fb64bd0a9d45111acc35f17 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:44.613857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de7ca3-1da5-4d3d-b0da-2ae266048f39 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:44.614400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de7ca3-1da5-4d3d-b0da-2ae266048f39 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:44.614400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de7ca3-1da5-4d3d-b0da-2ae266048f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:44.624720 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:44.639626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a1de7ca3-1da5-4d3d-b0da-2ae266048f39 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:44.640770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a1de7ca3-1da5-4d3d-b0da-2ae266048f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:44.727250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 999/2006] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:44.729717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72530c6e-a754-4060-b839-11d3f0307148 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:44.730220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72530c6e-a754-4060-b839-11d3f0307148 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:44.731175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72530c6e-a754-4060-b839-11d3f0307148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:44.752070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-72530c6e-a754-4060-b839-11d3f0307148 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:44.753331 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-72530c6e-a754-4060-b839-11d3f0307148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:44.763099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1008/2007] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:44.768456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de7ca3-1da5-4d3d-b0da-2ae266048f39 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:44.779816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de7ca3-1da5-4d3d-b0da-2ae266048f39 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:44.785424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797962f-d43c-446c-963b-8fdbcce33108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:44.786414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797962f-d43c-446c-963b-8fdbcce33108 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:44.786414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797962f-d43c-446c-963b-8fdbcce33108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:44.786680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0797962f-d43c-446c-963b-8fdbcce33108 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:44.787312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797962f-d43c-446c-963b-8fdbcce33108 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:44.787728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797962f-d43c-446c-963b-8fdbcce33108 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:44.789695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0797962f-d43c-446c-963b-8fdbcce33108 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:44.807283 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1000/2008] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:44.812062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72530c6e-a754-4060-b839-11d3f0307148 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:44.827857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72530c6e-a754-4060-b839-11d3f0307148 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:44.833442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6ebee-8343-49b2-9cbf-c746966f51f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:44.834146 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6ebee-8343-49b2-9cbf-c746966f51f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:44.834490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6ebee-8343-49b2-9cbf-c746966f51f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:44.834827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf6ebee-8343-49b2-9cbf-c746966f51f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:44.835577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6ebee-8343-49b2-9cbf-c746966f51f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:44.837586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf6ebee-8343-49b2-9cbf-c746966f51f2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:44.871866 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1009/2009] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:44 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:44.875843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797962f-d43c-446c-963b-8fdbcce33108 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:44.897565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0797962f-d43c-446c-963b-8fdbcce33108 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:44.982620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5500b-8592-4aaf-92cc-67fc34036f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:44.983231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5500b-8592-4aaf-92cc-67fc34036f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:44.983541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5500b-8592-4aaf-92cc-67fc34036f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:44.983816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5500b-8592-4aaf-92cc-67fc34036f2f admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:44.984467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5500b-8592-4aaf-92cc-67fc34036f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:44.986433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e5500b-8592-4aaf-92cc-67fc34036f2f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:45.012445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1001/2010] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:44 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:45.013790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6ebee-8343-49b2-9cbf-c746966f51f2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:45.024429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf6ebee-8343-49b2-9cbf-c746966f51f2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:45.029989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e677bff-4f07-45cf-bf85-512398f13341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:45.030596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e677bff-4f07-45cf-bf85-512398f13341 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:45.030871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e677bff-4f07-45cf-bf85-512398f13341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:45.031141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e677bff-4f07-45cf-bf85-512398f13341 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:45.031812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e677bff-4f07-45cf-bf85-512398f13341 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:45.033427 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e677bff-4f07-45cf-bf85-512398f13341 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:45.043650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1010/2011] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:45.048524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5500b-8592-4aaf-92cc-67fc34036f2f admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:45.062689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e5500b-8592-4aaf-92cc-67fc34036f2f admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:45.071471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6d86cb-650f-4d0b-bb97-6c4d8c114641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:45.072288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d86cb-650f-4d0b-bb97-6c4d8c114641 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:45.072629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d86cb-650f-4d0b-bb97-6c4d8c114641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:45.072973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6d86cb-650f-4d0b-bb97-6c4d8c114641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:45.073786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d86cb-650f-4d0b-bb97-6c4d8c114641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:45.077275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6d86cb-650f-4d0b-bb97-6c4d8c114641 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:45.094285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1002/2012] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:45.095491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e677bff-4f07-45cf-bf85-512398f13341 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:45.105818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e677bff-4f07-45cf-bf85-512398f13341 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:45.113663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5e6a1-bcde-4ca7-b159-0439fd9c6b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:45.113913 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5e6a1-bcde-4ca7-b159-0439fd9c6b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:45.114139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5e6a1-bcde-4ca7-b159-0439fd9c6b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:45.114409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5e6a1-bcde-4ca7-b159-0439fd9c6b28 admin admin] PATH_INFO: `/v3/projects/992cf5e5ceb945c6b13ed99c7a8a77ad/users/ec5e4b9ff50449a2b12519b0a1ddc065/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:45.133183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5e6a1-bcde-4ca7-b159-0439fd9c6b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=992cf5e5ceb945c6b13ed99c7a8a77ad, user_id=ec5e4b9ff50449a2b12519b0a1ddc065, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:45.135962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5e6a1-bcde-4ca7-b159-0439fd9c6b28 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:45.157169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1011/2013] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:45 2022] PUT /identity/v3/projects/992cf5e5ceb945c6b13ed99c7a8a77ad/users/ec5e4b9ff50449a2b12519b0a1ddc065/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:45.167270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52a52-c609-4959-8352-b632ed99aae6 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:45.167661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52a52-c609-4959-8352-b632ed99aae6 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:45.168113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e52a52-c609-4959-8352-b632ed99aae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:45.203624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-44e52a52-c609-4959-8352-b632ed99aae6 None None] MFA Rules not processed for user `ec5e4b9ff50449a2b12519b0a1ddc065`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:45.204898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-44e52a52-c609-4959-8352-b632ed99aae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:45.281715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:45.289988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1003/2014] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:45.300140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5e6a1-bcde-4ca7-b159-0439fd9c6b28 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:45.301941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b6b5e6a1-bcde-4ca7-b159-0439fd9c6b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:45.374198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5e6a1-bcde-4ca7-b159-0439fd9c6b28 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:45.378332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6695804-cc3a-4743-9989-f58c72c3a73f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:45.379119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6695804-cc3a-4743-9989-f58c72c3a73f service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:45.379809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6695804-cc3a-4743-9989-f58c72c3a73f service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:45.380200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6695804-cc3a-4743-9989-f58c72c3a73f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:45.390388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6695804-cc3a-4743-9989-f58c72c3a73f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:45.391921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6695804-cc3a-4743-9989-f58c72c3a73f service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:45.401772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1012/2015] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:47.350559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e52a52-c609-4959-8352-b632ed99aae6 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:47.352130 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-44e52a52-c609-4959-8352-b632ed99aae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:47.443646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e52a52-c609-4959-8352-b632ed99aae6 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:47.448305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:47.449095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:47.449502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:47.449879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] PATH_INFO: `/v3/users/fbda199975b04fa190689f1ac1d0cdff` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:47.451107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbda199975b04fa190689f1ac1d0cdff)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:47.453185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:47.504034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbda199975b04fa190689f1ac1d0cdff'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:47.504797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbda199975b04fa190689f1ac1d0cdff'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:47.513250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbda199975b04fa190689f1ac1d0cdff'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:47.520679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbda199975b04fa190689f1ac1d0cdff'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:47.521510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbda199975b04fa190689f1ac1d0cdff'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:47.540721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1004/2016] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:47 2022] DELETE /identity/v3/users/fbda199975b04fa190689f1ac1d0cdff => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:47.740867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6695804-cc3a-4743-9989-f58c72c3a73f service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:47.774818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6695804-cc3a-4743-9989-f58c72c3a73f service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:47.831997 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:47.832626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:47.832910 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:47.833175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] PATH_INFO: `/v3/projects/c9e09828a7e642618d6dd60e3bbaeb03` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:47.834278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9e09828a7e642618d6dd60e3bbaeb03)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:47.836236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:47.853306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:47.891785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9e09828a7e642618d6dd60e3bbaeb03', 'request_id': 'req-3597691e-a6c7-4169-b76d-80841540d834'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:47.897298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9e09828a7e642618d6dd60e3bbaeb03', 'request_id': 'req-3597691e-a6c7-4169-b76d-80841540d834'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:47.904625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9e09828a7e642618d6dd60e3bbaeb03', 'request_id': 'req-3597691e-a6c7-4169-b76d-80841540d834'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:47.916073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] The token cache is being invalidate because project c9e09828a7e642618d6dd60e3bbaeb03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:47.916361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:47.917032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:47.918815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1013/2017] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:47 2022] DELETE /identity/v3/projects/c9e09828a7e642618d6dd60e3bbaeb03 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:47.922812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:47.924346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:47.937776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6884e3-a079-4f57-af00-17282d7d025d admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:48.000751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:48.001331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.001612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.001888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] PATH_INFO: `/v3/users/226c04e7d4ef407d829867bcc77e38f9` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.003141 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226c04e7d4ef407d829867bcc77e38f9)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:48.005123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:48.026830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:48.052352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226c04e7d4ef407d829867bcc77e38f9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.052995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226c04e7d4ef407d829867bcc77e38f9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.060647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226c04e7d4ef407d829867bcc77e38f9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.067900 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226c04e7d4ef407d829867bcc77e38f9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.068544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226c04e7d4ef407d829867bcc77e38f9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.096937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:48.098285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1005/2018] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:47 2022] DELETE /identity/v3/users/226c04e7d4ef407d829867bcc77e38f9 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:48.098415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:48.112297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597691e-a6c7-4169-b76d-80841540d834 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:48.188358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:48.189074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.189350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.189618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] PATH_INFO: `/v3/users/ec5e4b9ff50449a2b12519b0a1ddc065` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.190821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec5e4b9ff50449a2b12519b0a1ddc065)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:48.192772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:48.251870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec5e4b9ff50449a2b12519b0a1ddc065'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.259545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec5e4b9ff50449a2b12519b0a1ddc065'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.260213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec5e4b9ff50449a2b12519b0a1ddc065'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.260886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec5e4b9ff50449a2b12519b0a1ddc065'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.270039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b63bca4e-cb4f-4f2f-a832-c2f5141cb6b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec5e4b9ff50449a2b12519b0a1ddc065'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.295033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1014/2019] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:14:48 2022] DELETE /identity/v3/users/ec5e4b9ff50449a2b12519b0a1ddc065 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:48.298286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:48.298851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8e348-a442-4bf9-8486-db4d73008ccc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.299140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8e348-a442-4bf9-8486-db4d73008ccc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.299411 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea8e348-a442-4bf9-8486-db4d73008ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.300064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:48.316470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75932717-196e-4e8f-b0be-f513cff21c11 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:48.321418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8ea8e348-a442-4bf9-8486-db4d73008ccc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:48.323691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8ea8e348-a442-4bf9-8486-db4d73008ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:48.345114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:48.385752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:48.387986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.388435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.388875 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] PATH_INFO: `/v3/projects/b06b34af70ca40328ec4480d4558aec7` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.390231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b06b34af70ca40328ec4480d4558aec7)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:48.392185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:48.399392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1015/2020] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:48.410237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8e348-a442-4bf9-8486-db4d73008ccc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:48.411697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8ea8e348-a442-4bf9-8486-db4d73008ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:48.444154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b06b34af70ca40328ec4480d4558aec7', 'request_id': 'req-5ec23b7b-5284-445e-a664-06514bfcdf1b'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.453039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b06b34af70ca40328ec4480d4558aec7', 'request_id': 'req-5ec23b7b-5284-445e-a664-06514bfcdf1b'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.459355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b06b34af70ca40328ec4480d4558aec7', 'request_id': 'req-5ec23b7b-5284-445e-a664-06514bfcdf1b'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.472661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] The token cache is being invalidate because project b06b34af70ca40328ec4480d4558aec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:48.472986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.473607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5ec23b7b-5284-445e-a664-06514bfcdf1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:48.475350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1006/2021] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:48 2022] DELETE /identity/v3/projects/b06b34af70ca40328ec4480d4558aec7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:48.480174 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91b0e1-56b1-4c3b-8019-d442289f0ee9 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.480756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91b0e1-56b1-4c3b-8019-d442289f0ee9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.480975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b91b0e1-56b1-4c3b-8019-d442289f0ee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.486474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea8e348-a442-4bf9-8486-db4d73008ccc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:48.504816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-9b91b0e1-56b1-4c3b-8019-d442289f0ee9 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:48.505988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9b91b0e1-56b1-4c3b-8019-d442289f0ee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:48.538156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:48.558961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541945f-868f-4a23-8a2e-048660249ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:48.559603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541945f-868f-4a23-8a2e-048660249ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.559886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541945f-868f-4a23-8a2e-048660249ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.560161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1541945f-868f-4a23-8a2e-048660249ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.561942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1541945f-868f-4a23-8a2e-048660249ab4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:48.588392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1007/2022] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:48.595996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d577-8a9b-41fb-afbd-3952273b5ad6 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.596564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d577-8a9b-41fb-afbd-3952273b5ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.596906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c6d577-8a9b-41fb-afbd-3952273b5ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.623907 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d0c6d577-8a9b-41fb-afbd-3952273b5ad6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:48.625396 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d0c6d577-8a9b-41fb-afbd-3952273b5ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:48.634477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1008/2023] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:48.636533 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541945f-868f-4a23-8a2e-048660249ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:48.638731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1541945f-868f-4a23-8a2e-048660249ab4 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:48.639521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d577-8a9b-41fb-afbd-3952273b5ad6 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:48.649686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1016/2024] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:14:48.664933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c6d577-8a9b-41fb-afbd-3952273b5ad6 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:48.671196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107014c4-6380-4ad1-85aa-963420525b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:48.671918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107014c4-6380-4ad1-85aa-963420525b48 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.672248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107014c4-6380-4ad1-85aa-963420525b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.672586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107014c4-6380-4ad1-85aa-963420525b48 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.673346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107014c4-6380-4ad1-85aa-963420525b48 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:48.673707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107014c4-6380-4ad1-85aa-963420525b48 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:48.675368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107014c4-6380-4ad1-85aa-963420525b48 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:48.687691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1009/2025] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:48.692351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541945f-868f-4a23-8a2e-048660249ab4 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:48.702806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1541945f-868f-4a23-8a2e-048660249ab4 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:48.707662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff490a8-19f4-4b3d-92cc-25c88923c6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:48.708259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff490a8-19f4-4b3d-92cc-25c88923c6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.708530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff490a8-19f4-4b3d-92cc-25c88923c6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.708795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff490a8-19f4-4b3d-92cc-25c88923c6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.709444 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff490a8-19f4-4b3d-92cc-25c88923c6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:48.711333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff490a8-19f4-4b3d-92cc-25c88923c6f1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:48.740921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1017/2026] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:48 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:48.745073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107014c4-6380-4ad1-85aa-963420525b48 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:48.755792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107014c4-6380-4ad1-85aa-963420525b48 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:48.821005 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1d5ae-7aeb-4d0a-bf28-75ef248466ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:48.821610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1d5ae-7aeb-4d0a-bf28-75ef248466ce admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.821880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1d5ae-7aeb-4d0a-bf28-75ef248466ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.822140 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f1d5ae-7aeb-4d0a-bf28-75ef248466ce admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.822761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1d5ae-7aeb-4d0a-bf28-75ef248466ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:48.825782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f1d5ae-7aeb-4d0a-bf28-75ef248466ce admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:48.849956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1010/2027] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:48 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:48.854949 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff490a8-19f4-4b3d-92cc-25c88923c6f1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:48.868597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff490a8-19f4-4b3d-92cc-25c88923c6f1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:48.873592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117e74-e24b-4325-8106-10ab9fcbdc17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:48.874587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117e74-e24b-4325-8106-10ab9fcbdc17 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.874587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117e74-e24b-4325-8106-10ab9fcbdc17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.874782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70117e74-e24b-4325-8106-10ab9fcbdc17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.875468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70117e74-e24b-4325-8106-10ab9fcbdc17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:48.877226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70117e74-e24b-4325-8106-10ab9fcbdc17 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:48.878981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:48.891010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1018/2028] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:48.897568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1d5ae-7aeb-4d0a-bf28-75ef248466ce admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:48.910267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f1d5ae-7aeb-4d0a-bf28-75ef248466ce admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:48.915162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a453f679-5745-4caa-b3e7-80baa6f4f284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:48.915802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453f679-5745-4caa-b3e7-80baa6f4f284 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.916073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453f679-5745-4caa-b3e7-80baa6f4f284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.916336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a453f679-5745-4caa-b3e7-80baa6f4f284 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.917099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453f679-5745-4caa-b3e7-80baa6f4f284 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:48.919209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a453f679-5745-4caa-b3e7-80baa6f4f284 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:48.933456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1011/2029] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:48.935302 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117e74-e24b-4325-8106-10ab9fcbdc17 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:48.946074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70117e74-e24b-4325-8106-10ab9fcbdc17 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:48.953245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320801f5-7c44-490c-a644-6c0bba8b8d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:48.953983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320801f5-7c44-490c-a644-6c0bba8b8d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:48.954288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320801f5-7c44-490c-a644-6c0bba8b8d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:48.954517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320801f5-7c44-490c-a644-6c0bba8b8d29 admin admin] PATH_INFO: `/v3/projects/7fbd70583607465bb40c6a6d5bdc832a/users/1f6edcbf6b5b4689a39f0ed03ca4c623/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:48.973634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320801f5-7c44-490c-a644-6c0bba8b8d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fbd70583607465bb40c6a6d5bdc832a, user_id=1f6edcbf6b5b4689a39f0ed03ca4c623, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:48.977221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320801f5-7c44-490c-a644-6c0bba8b8d29 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:48.996854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1019/2030] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:48 2022] PUT /identity/v3/projects/7fbd70583607465bb40c6a6d5bdc832a/users/1f6edcbf6b5b4689a39f0ed03ca4c623/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:49.008458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91f488-67f9-4411-b448-f694d6fe5b5e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.008458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91f488-67f9-4411-b448-f694d6fe5b5e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.008781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb91f488-67f9-4411-b448-f694d6fe5b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.049736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320801f5-7c44-490c-a644-6c0bba8b8d29 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.053540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-320801f5-7c44-490c-a644-6c0bba8b8d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:49.065671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-cb91f488-67f9-4411-b448-f694d6fe5b5e None None] MFA Rules not processed for user `1f6edcbf6b5b4689a39f0ed03ca4c623`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:49.067132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-cb91f488-67f9-4411-b448-f694d6fe5b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:49.159650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1012/2031] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:49.163971 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91f488-67f9-4411-b448-f694d6fe5b5e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.176707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320801f5-7c44-490c-a644-6c0bba8b8d29 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.181109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb91f488-67f9-4411-b448-f694d6fe5b5e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.182237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.182955 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.183235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.183616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] PATH_INFO: `/v3/projects/992cf5e5ceb945c6b13ed99c7a8a77ad` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.184821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=992cf5e5ceb945c6b13ed99c7a8a77ad)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.186772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.188032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb641ce-2c4e-4d2e-8c74-68b59c03bf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.188948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb641ce-2c4e-4d2e-8c74-68b59c03bf90 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.189452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb641ce-2c4e-4d2e-8c74-68b59c03bf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.189823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb641ce-2c4e-4d2e-8c74-68b59c03bf90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.191642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb641ce-2c4e-4d2e-8c74-68b59c03bf90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.193770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb641ce-2c4e-4d2e-8c74-68b59c03bf90 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.219989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:49.244979 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1013/2032] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:49 2022] POST /identity/v3/projects => generated 383 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:49.252461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb641ce-2c4e-4d2e-8c74-68b59c03bf90 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.296550 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '992cf5e5ceb945c6b13ed99c7a8a77ad', 'request_id': 'req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:49.303925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '992cf5e5ceb945c6b13ed99c7a8a77ad', 'request_id': 'req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:49.313474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '992cf5e5ceb945c6b13ed99c7a8a77ad', 'request_id': 'req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:49.326571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] The token cache is being invalidate because project 992cf5e5ceb945c6b13ed99c7a8a77ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:14:49.326996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:49.328081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-798f08a7-7564-4eb9-a1fc-9f7da12c15f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:14:49.329857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1020/2033] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:14:49 2022] DELETE /identity/v3/projects/992cf5e5ceb945c6b13ed99c7a8a77ad => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:49.334507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603677fb-0361-4161-b1e7-073eacf79b5e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.334790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603677fb-0361-4161-b1e7-073eacf79b5e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.335065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603677fb-0361-4161-b1e7-073eacf79b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.343861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb641ce-2c4e-4d2e-8c74-68b59c03bf90 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.358508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-603677fb-0361-4161-b1e7-073eacf79b5e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:49.359884 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-603677fb-0361-4161-b1e7-073eacf79b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:49.411360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54c07e-682a-47a8-ad46-1026f2aea6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.412817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54c07e-682a-47a8-ad46-1026f2aea6d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.413556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54c07e-682a-47a8-ad46-1026f2aea6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.413835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a54c07e-682a-47a8-ad46-1026f2aea6d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.415804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54c07e-682a-47a8-ad46-1026f2aea6d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.420901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a54c07e-682a-47a8-ad46-1026f2aea6d9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.436795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1021/2034] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:49.444752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9aa59-ad87-4a90-8e9a-ead3d472d19d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.445043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9aa59-ad87-4a90-8e9a-ead3d472d19d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.445324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9aa59-ad87-4a90-8e9a-ead3d472d19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.454306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1014/2035] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:49 2022] POST /identity/v3/users => generated 298 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:14:49.455580 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:49.458377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54c07e-682a-47a8-ad46-1026f2aea6d9 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.460923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3a54c07e-682a-47a8-ad46-1026f2aea6d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:49.471653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-36a9aa59-ad87-4a90-8e9a-ead3d472d19d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:49.473354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-36a9aa59-ad87-4a90-8e9a-ead3d472d19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:49.475690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a54c07e-682a-47a8-ad46-1026f2aea6d9 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.481562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753a9f-9ca7-4dd4-9b2f-69b926f5c83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.482598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753a9f-9ca7-4dd4-9b2f-69b926f5c83e admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.482881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753a9f-9ca7-4dd4-9b2f-69b926f5c83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.483150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753a9f-9ca7-4dd4-9b2f-69b926f5c83e admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.483833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753a9f-9ca7-4dd4-9b2f-69b926f5c83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.484356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1022/2036] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:14:49.487713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753a9f-9ca7-4dd4-9b2f-69b926f5c83e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.488773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9aa59-ad87-4a90-8e9a-ead3d472d19d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.499509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1015/2037] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:49.499690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9aa59-ad87-4a90-8e9a-ead3d472d19d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.501941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753a9f-9ca7-4dd4-9b2f-69b926f5c83e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.504026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27398-3bc0-49e2-a840-96e59d673aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.504625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27398-3bc0-49e2-a840-96e59d673aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.504899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27398-3bc0-49e2-a840-96e59d673aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.505658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa27398-3bc0-49e2-a840-96e59d673aa8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.505797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27398-3bc0-49e2-a840-96e59d673aa8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:14:49.506101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27398-3bc0-49e2-a840-96e59d673aa8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.507744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa27398-3bc0-49e2-a840-96e59d673aa8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.514509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753a9f-9ca7-4dd4-9b2f-69b926f5c83e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.520324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1c626-5926-4027-af35-f38e82bab894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.520982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1c626-5926-4027-af35-f38e82bab894 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.521252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1c626-5926-4027-af35-f38e82bab894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.521517 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1c626-5926-4027-af35-f38e82bab894 admin admin] PATH_INFO: `/v3/projects/c3e35afdd88b4fb4be52e223d759960d/users/d02d0c6e1cab4073a914a1d7371929ab/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.523481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1023/2038] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:14:49.526375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27398-3bc0-49e2-a840-96e59d673aa8 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.537535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa27398-3bc0-49e2-a840-96e59d673aa8 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.538041 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1c626-5926-4027-af35-f38e82bab894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e35afdd88b4fb4be52e223d759960d, user_id=d02d0c6e1cab4073a914a1d7371929ab, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.540799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1c626-5926-4027-af35-f38e82bab894 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.542521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8758912-44f7-454e-9441-c1128b3eb062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.544094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8758912-44f7-454e-9441-c1128b3eb062 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.544591 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8758912-44f7-454e-9441-c1128b3eb062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.544873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8758912-44f7-454e-9441-c1128b3eb062 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.545645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8758912-44f7-454e-9441-c1128b3eb062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.547938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8758912-44f7-454e-9441-c1128b3eb062 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.557680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1016/2039] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:49 2022] PUT /identity/v3/projects/c3e35afdd88b4fb4be52e223d759960d/users/d02d0c6e1cab4073a914a1d7371929ab/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:49.561459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1c626-5926-4027-af35-f38e82bab894 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.582178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1024/2040] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:14:49 2022] POST /identity/v3/projects => generated 397 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:49.586452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8758912-44f7-454e-9441-c1128b3eb062 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.634411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:49.640598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1c626-5926-4027-af35-f38e82bab894 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.664236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8758912-44f7-454e-9441-c1128b3eb062 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.670889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e41fa0-39e4-4bd7-8701-b4f475a1e744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.671811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e41fa0-39e4-4bd7-8701-b4f475a1e744 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.671993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e41fa0-39e4-4bd7-8701-b4f475a1e744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.672252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e41fa0-39e4-4bd7-8701-b4f475a1e744 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.674075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e41fa0-39e4-4bd7-8701-b4f475a1e744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.675809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e41fa0-39e4-4bd7-8701-b4f475a1e744 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.703539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1025/2041] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:14:49 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:14:49.708419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e41fa0-39e4-4bd7-8701-b4f475a1e744 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.712245 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686dd86-cf23-4cb2-a884-c866818f7b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.713020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686dd86-cf23-4cb2-a884-c866818f7b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.713300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686dd86-cf23-4cb2-a884-c866818f7b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.713574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686dd86-cf23-4cb2-a884-c866818f7b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.714230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686dd86-cf23-4cb2-a884-c866818f7b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.717796 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686dd86-cf23-4cb2-a884-c866818f7b96 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.721539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e41fa0-39e4-4bd7-8701-b4f475a1e744 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.727517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e555f-5a66-43d2-b7d6-b41d14846801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.728204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e555f-5a66-43d2-b7d6-b41d14846801 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.728555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e555f-5a66-43d2-b7d6-b41d14846801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.728899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e555f-5a66-43d2-b7d6-b41d14846801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.729606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e555f-5a66-43d2-b7d6-b41d14846801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.734447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1017/2042] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:49.734611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e555f-5a66-43d2-b7d6-b41d14846801 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.735625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686dd86-cf23-4cb2-a884-c866818f7b96 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.748298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686dd86-cf23-4cb2-a884-c866818f7b96 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.749314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1026/2043] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:14:49.752599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e555f-5a66-43d2-b7d6-b41d14846801 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.756385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4043d69e-9653-4d4e-9609-64b9b0d343bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.757061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043d69e-9653-4d4e-9609-64b9b0d343bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.757341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043d69e-9653-4d4e-9609-64b9b0d343bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.757612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4043d69e-9653-4d4e-9609-64b9b0d343bf admin admin] PATH_INFO: `/v3/projects/c3e35afdd88b4fb4be52e223d759960d/users/d02d0c6e1cab4073a914a1d7371929ab/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.760511 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043d69e-9653-4d4e-9609-64b9b0d343bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3e35afdd88b4fb4be52e223d759960d, user_id=d02d0c6e1cab4073a914a1d7371929ab, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.764104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4043d69e-9653-4d4e-9609-64b9b0d343bf admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.764933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e555f-5a66-43d2-b7d6-b41d14846801 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.769776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65e16b-209f-43b1-bd05-b97843f84e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.770545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e16b-209f-43b1-bd05-b97843f84e18 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.771073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e16b-209f-43b1-bd05-b97843f84e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.771348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b65e16b-209f-43b1-bd05-b97843f84e18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.772126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65e16b-209f-43b1-bd05-b97843f84e18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.774044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b65e16b-209f-43b1-bd05-b97843f84e18 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.779692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1018/2044] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:49 2022] PUT /identity/v3/projects/c3e35afdd88b4fb4be52e223d759960d/users/d02d0c6e1cab4073a914a1d7371929ab/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:14:49.786606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1027/2045] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:14:49.789981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35bc37-4e30-4e8b-9c19-d0068c186738 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.789981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35bc37-4e30-4e8b-9c19-d0068c186738 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.790197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf35bc37-4e30-4e8b-9c19-d0068c186738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.790280 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65e16b-209f-43b1-bd05-b97843f84e18 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.804661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b65e16b-209f-43b1-bd05-b97843f84e18 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:49.828521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-bf35bc37-4e30-4e8b-9c19-d0068c186738 None None] MFA Rules not processed for user `d02d0c6e1cab4073a914a1d7371929ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:49.829833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bf35bc37-4e30-4e8b-9c19-d0068c186738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:49.874668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f5b139-bdf8-485c-b936-adff8cad7d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:49.875342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5b139-bdf8-485c-b936-adff8cad7d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.875654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5b139-bdf8-485c-b936-adff8cad7d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.875933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5b139-bdf8-485c-b936-adff8cad7d85 admin admin] PATH_INFO: `/v3/projects/4ccd70c12f81471cad0bd8feae2f5d25/users/eaf6fc5f041f4d8b8d4599c71e0b79e2/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.895699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5b139-bdf8-485c-b936-adff8cad7d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ccd70c12f81471cad0bd8feae2f5d25, user_id=eaf6fc5f041f4d8b8d4599c71e0b79e2, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:49.900401 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f5b139-bdf8-485c-b936-adff8cad7d85 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:49.922407 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1019/2046] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:49.936504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35bc37-4e30-4e8b-9c19-d0068c186738 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:49.940079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1028/2047] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:14:49 2022] PUT /identity/v3/projects/4ccd70c12f81471cad0bd8feae2f5d25/users/eaf6fc5f041f4d8b8d4599c71e0b79e2/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:14:49.940712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bf35bc37-4e30-4e8b-9c19-d0068c186738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:49.944907 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49d1d5-e718-4516-8e5e-5c93c7b8b582 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:49.945469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49d1d5-e718-4516-8e5e-5c93c7b8b582 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:49.945960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a49d1d5-e718-4516-8e5e-5c93c7b8b582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:49.978794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:50.004624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1a49d1d5-e718-4516-8e5e-5c93c7b8b582 None None] MFA Rules not processed for user `eaf6fc5f041f4d8b8d4599c71e0b79e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:14:50.006585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1a49d1d5-e718-4516-8e5e-5c93c7b8b582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:50.041804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf35bc37-4e30-4e8b-9c19-d0068c186738 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:50.081865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:14:50.105745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1029/2048] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:14:50.117562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6614a4d-179d-4157-aaf9-cdbbd627f2a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:50.122840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49d1d5-e718-4516-8e5e-5c93c7b8b582 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:14:50.124692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1a49d1d5-e718-4516-8e5e-5c93c7b8b582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:50.126234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6614a4d-179d-4157-aaf9-cdbbd627f2a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:50.127605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6614a4d-179d-4157-aaf9-cdbbd627f2a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:50.128181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6614a4d-179d-4157-aaf9-cdbbd627f2a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:50.135029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b6614a4d-179d-4157-aaf9-cdbbd627f2a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:14:50.142732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a49d1d5-e718-4516-8e5e-5c93c7b8b582 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:14:50.146632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cc685-7ce3-4a9f-a30a-b00d94c166c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:14:50.147310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cc685-7ce3-4a9f-a30a-b00d94c166c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:14:50.147614 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cc685-7ce3-4a9f-a30a-b00d94c166c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:14:50.147885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5cc685-7ce3-4a9f-a30a-b00d94c166c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:14:50.159286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cc685-7ce3-4a9f-a30a-b00d94c166c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:50.161445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5cc685-7ce3-4a9f-a30a-b00d94c166c2 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:50.173591 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1030/2049] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:50 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:14:50.225968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6614a4d-179d-4157-aaf9-cdbbd627f2a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:14:50.228443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6614a4d-179d-4157-aaf9-cdbbd627f2a8 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:14:50.241220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1020/2050] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:14:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:00.166520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=33 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:00.235649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=35 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:00.344996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cc685-7ce3-4a9f-a30a-b00d94c166c2 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:00.353445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5cc685-7ce3-4a9f-a30a-b00d94c166c2 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:00.357996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d0574-ec51-4152-8222-ab8535c5ab86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:00.358577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d0574-ec51-4152-8222-ab8535c5ab86 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:00.358851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d0574-ec51-4152-8222-ab8535c5ab86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:00.359122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d0574-ec51-4152-8222-ab8535c5ab86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:00.417234 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d0574-ec51-4152-8222-ab8535c5ab86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:00.419223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2d0574-ec51-4152-8222-ab8535c5ab86 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:00.429311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1031/2051] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:00 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:00.804424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6614a4d-179d-4157-aaf9-cdbbd627f2a8 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:00.877453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6614a4d-179d-4157-aaf9-cdbbd627f2a8 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:00.882588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:00.883246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:00.883552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:00.883821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] PATH_INFO: `/v3/users/eaf6fc5f041f4d8b8d4599c71e0b79e2` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:00.884965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf6fc5f041f4d8b8d4599c71e0b79e2)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:00.887979 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:00.951152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf6fc5f041f4d8b8d4599c71e0b79e2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:00.951862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf6fc5f041f4d8b8d4599c71e0b79e2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:00.960097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf6fc5f041f4d8b8d4599c71e0b79e2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:00.968109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf6fc5f041f4d8b8d4599c71e0b79e2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:00.968872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf6fc5f041f4d8b8d4599c71e0b79e2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:00.989430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1021/2052] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:00 2022] DELETE /identity/v3/users/eaf6fc5f041f4d8b8d4599c71e0b79e2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:00.993979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6529fe-a45a-4d90-9b87-53420301f61f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:00.994262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6529fe-a45a-4d90-9b87-53420301f61f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:00.994537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6529fe-a45a-4d90-9b87-53420301f61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:01.014871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3a6529fe-a45a-4d90-9b87-53420301f61f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:01.016071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3a6529fe-a45a-4d90-9b87-53420301f61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:01.095484 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1032/2053] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:01.096251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:01.097716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:01.172495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394d06db-9d42-4ecb-915a-d0d7bc6e7025 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:01.177510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306a0d9-8201-4af4-925d-22c814ea6817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:01.178229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306a0d9-8201-4af4-925d-22c814ea6817 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:01.178573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306a0d9-8201-4af4-925d-22c814ea6817 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:01.178913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306a0d9-8201-4af4-925d-22c814ea6817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:01.194497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306a0d9-8201-4af4-925d-22c814ea6817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:01.196189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7306a0d9-8201-4af4-925d-22c814ea6817 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:01.208570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1022/2054] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:01.581035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6529fe-a45a-4d90-9b87-53420301f61f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:01.582725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3a6529fe-a45a-4d90-9b87-53420301f61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:01.604422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6529fe-a45a-4d90-9b87-53420301f61f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:01.612221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:01.613460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:01.613832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:01.614175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] PATH_INFO: `/v3/projects/4ccd70c12f81471cad0bd8feae2f5d25` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:01.615649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ccd70c12f81471cad0bd8feae2f5d25)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:01.619756 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:01.700996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ccd70c12f81471cad0bd8feae2f5d25', 'request_id': 'req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:01.709026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ccd70c12f81471cad0bd8feae2f5d25', 'request_id': 'req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:01.725061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ccd70c12f81471cad0bd8feae2f5d25', 'request_id': 'req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:01.757735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] The token cache is being invalidate because project 4ccd70c12f81471cad0bd8feae2f5d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:01.758291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:01.759171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-29907604-cbd2-4b7c-9ae5-49e45b1fd6dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:01.781890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1033/2055] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:01 2022] DELETE /identity/v3/projects/4ccd70c12f81471cad0bd8feae2f5d25 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:05.500400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306a0d9-8201-4af4-925d-22c814ea6817 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:05.501788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7306a0d9-8201-4af4-925d-22c814ea6817 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:05.515382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306a0d9-8201-4af4-925d-22c814ea6817 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:05.535727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:05.577049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:05.577700 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:05.578028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:05.578293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] PATH_INFO: `/v3/users/1f6edcbf6b5b4689a39f0ed03ca4c623` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:05.579569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f6edcbf6b5b4689a39f0ed03ca4c623)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:05.581592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:05.633975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f6edcbf6b5b4689a39f0ed03ca4c623'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:05.634920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f6edcbf6b5b4689a39f0ed03ca4c623'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:05.647929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f6edcbf6b5b4689a39f0ed03ca4c623'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:05.656747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f6edcbf6b5b4689a39f0ed03ca4c623'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:05.657477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f6edcbf6b5b4689a39f0ed03ca4c623'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:05.677114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1023/2056] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:05 2022] DELETE /identity/v3/users/1f6edcbf6b5b4689a39f0ed03ca4c623 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:05.681361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affbedd-81b1-4ebb-abce-c481449d0fc0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:05.681857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affbedd-81b1-4ebb-abce-c481449d0fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:05.681950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affbedd-81b1-4ebb-abce-c481449d0fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:05.702607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-9affbedd-81b1-4ebb-abce-c481449d0fc0 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:05.704057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9affbedd-81b1-4ebb-abce-c481449d0fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:05.719976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:05.774371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1034/2057] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:05.784409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:05.785960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:05.847993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7214dfd-cb3b-43ec-9f8a-f8df49b988f3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:05.851735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cf1de-1177-4fbb-a9a9-c00441c4a8ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:05.852342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cf1de-1177-4fbb-a9a9-c00441c4a8ae service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:05.852840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cf1de-1177-4fbb-a9a9-c00441c4a8ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:05.853213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cf1de-1177-4fbb-a9a9-c00441c4a8ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:05.865731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cf1de-1177-4fbb-a9a9-c00441c4a8ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:05.867161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cf1de-1177-4fbb-a9a9-c00441c4a8ae service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:05.876932 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1024/2058] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:06.208441 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affbedd-81b1-4ebb-abce-c481449d0fc0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:06.209872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9affbedd-81b1-4ebb-abce-c481449d0fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:06.227419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affbedd-81b1-4ebb-abce-c481449d0fc0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:06.233342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:06.233975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:06.234295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:06.234508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] PATH_INFO: `/v3/projects/7fbd70583607465bb40c6a6d5bdc832a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:06.235755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fbd70583607465bb40c6a6d5bdc832a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:06.237736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:06.283380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fbd70583607465bb40c6a6d5bdc832a', 'request_id': 'req-86343b3f-ca0e-4807-8999-97fa74a9def0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.288847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fbd70583607465bb40c6a6d5bdc832a', 'request_id': 'req-86343b3f-ca0e-4807-8999-97fa74a9def0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.296478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fbd70583607465bb40c6a6d5bdc832a', 'request_id': 'req-86343b3f-ca0e-4807-8999-97fa74a9def0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.307528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] The token cache is being invalidate because project 7fbd70583607465bb40c6a6d5bdc832a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:06.307821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.308496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.310785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1035/2059] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:06 2022] DELETE /identity/v3/projects/7fbd70583607465bb40c6a6d5bdc832a => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:06.314741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cf1de-1177-4fbb-a9a9-c00441c4a8ae service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:06.325355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cf1de-1177-4fbb-a9a9-c00441c4a8ae service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:06.372775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:06.388353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:06.388972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:06.389252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:06.389526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] PATH_INFO: `/v3/users/d02d0c6e1cab4073a914a1d7371929ab` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:06.390637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d02d0c6e1cab4073a914a1d7371929ab)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:06.392648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:06.440408 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd02d0c6e1cab4073a914a1d7371929ab'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.441121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd02d0c6e1cab4073a914a1d7371929ab'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.447864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd02d0c6e1cab4073a914a1d7371929ab'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.454882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd02d0c6e1cab4073a914a1d7371929ab'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.455543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6ee0a54a-4375-40df-b5a6-c40cb08d0f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd02d0c6e1cab4073a914a1d7371929ab'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.473564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1025/2060] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:06 2022] DELETE /identity/v3/users/d02d0c6e1cab4073a914a1d7371929ab => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:06.750292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:06.751774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:06.765371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86343b3f-ca0e-4807-8999-97fa74a9def0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:06.822116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:06.822670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:06.822950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:06.823223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] PATH_INFO: `/v3/projects/c3e35afdd88b4fb4be52e223d759960d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:06.824375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3e35afdd88b4fb4be52e223d759960d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:06.826987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:06.874780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:06.879171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3e35afdd88b4fb4be52e223d759960d', 'request_id': 'req-861d6123-208f-4d74-ae6a-2cf83d2bbf22'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.886506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3e35afdd88b4fb4be52e223d759960d', 'request_id': 'req-861d6123-208f-4d74-ae6a-2cf83d2bbf22'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.894041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3e35afdd88b4fb4be52e223d759960d', 'request_id': 'req-861d6123-208f-4d74-ae6a-2cf83d2bbf22'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.905692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] The token cache is being invalidate because project c3e35afdd88b4fb4be52e223d759960d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:06.905996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.906645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-861d6123-208f-4d74-ae6a-2cf83d2bbf22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:06.908197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1036/2061] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:06 2022] DELETE /identity/v3/projects/c3e35afdd88b4fb4be52e223d759960d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:06.914227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29d9ab-12a5-4617-a48b-f32231b5965e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:06.914708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29d9ab-12a5-4617-a48b-f32231b5965e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:06.914927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae29d9ab-12a5-4617-a48b-f32231b5965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:06.936701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-ae29d9ab-12a5-4617-a48b-f32231b5965e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:06.937981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ae29d9ab-12a5-4617-a48b-f32231b5965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:07.005805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1026/2062] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:07.015146 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a8c98-1077-484b-8901-850314ae5958 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.015433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a8c98-1077-484b-8901-850314ae5958 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.015740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a8c98-1077-484b-8901-850314ae5958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.035505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b05a8c98-1077-484b-8901-850314ae5958 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:07.036718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b05a8c98-1077-484b-8901-850314ae5958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:07.045414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1037/2063] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:07.049388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29d9ab-12a5-4617-a48b-f32231b5965e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.058795 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae29d9ab-12a5-4617-a48b-f32231b5965e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.063948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d8d29-1783-4ca0-9f7e-c3235917404b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.064539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d8d29-1783-4ca0-9f7e-c3235917404b admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.064815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d8d29-1783-4ca0-9f7e-c3235917404b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.065087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24d8d29-1783-4ca0-9f7e-c3235917404b admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.065728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d8d29-1783-4ca0-9f7e-c3235917404b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:15:07.066031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d8d29-1783-4ca0-9f7e-c3235917404b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.067665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24d8d29-1783-4ca0-9f7e-c3235917404b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.079035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1027/2064] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:15:07.083373 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a8c98-1077-484b-8901-850314ae5958 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.093382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a8c98-1077-484b-8901-850314ae5958 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.098343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484bf4b-8293-41e5-a4ad-2fbc33878b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.098941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484bf4b-8293-41e5-a4ad-2fbc33878b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.099215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484bf4b-8293-41e5-a4ad-2fbc33878b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.099510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484bf4b-8293-41e5-a4ad-2fbc33878b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.100135 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484bf4b-8293-41e5-a4ad-2fbc33878b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.102047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484bf4b-8293-41e5-a4ad-2fbc33878b7e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.130098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1038/2065] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:15:07 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:07.134430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d8d29-1783-4ca0-9f7e-c3235917404b admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.145149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24d8d29-1783-4ca0-9f7e-c3235917404b admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.194888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:07.207293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b650b-bb8b-479c-824a-102acf7e5e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.207959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b650b-bb8b-479c-824a-102acf7e5e88 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.208237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b650b-bb8b-479c-824a-102acf7e5e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.208538 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82b650b-bb8b-479c-824a-102acf7e5e88 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.209166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b650b-bb8b-479c-824a-102acf7e5e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.211539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82b650b-bb8b-479c-824a-102acf7e5e88 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.235954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1028/2066] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:15:07 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:07.240144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484bf4b-8293-41e5-a4ad-2fbc33878b7e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.250898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484bf4b-8293-41e5-a4ad-2fbc33878b7e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.255974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4abd8-8f47-443f-9516-d46251fe690f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.256582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4abd8-8f47-443f-9516-d46251fe690f admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.256862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4abd8-8f47-443f-9516-d46251fe690f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.257136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be4abd8-8f47-443f-9516-d46251fe690f admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.257815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4abd8-8f47-443f-9516-d46251fe690f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.259421 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be4abd8-8f47-443f-9516-d46251fe690f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.270121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1039/2067] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:07.273977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b650b-bb8b-479c-824a-102acf7e5e88 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.285051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82b650b-bb8b-479c-824a-102acf7e5e88 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.289460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1299cd-fbff-451f-b4fe-b51bb66ccae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.290063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1299cd-fbff-451f-b4fe-b51bb66ccae1 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.290368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1299cd-fbff-451f-b4fe-b51bb66ccae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.290637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1299cd-fbff-451f-b4fe-b51bb66ccae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.291316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1299cd-fbff-451f-b4fe-b51bb66ccae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.292983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1299cd-fbff-451f-b4fe-b51bb66ccae1 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.304912 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1029/2068] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:15:07.309876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4abd8-8f47-443f-9516-d46251fe690f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.320114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be4abd8-8f47-443f-9516-d46251fe690f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.325262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2865767-6604-4afc-a2f4-09f635f74d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.326168 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2865767-6604-4afc-a2f4-09f635f74d14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.326440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2865767-6604-4afc-a2f4-09f635f74d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.326719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2865767-6604-4afc-a2f4-09f635f74d14 admin admin] PATH_INFO: `/v3/projects/41a4d8fd02b2461a97110417ebf18745/users/6ff4a397043a48caa554a094d360fc1c/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.341107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2865767-6604-4afc-a2f4-09f635f74d14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a4d8fd02b2461a97110417ebf18745, user_id=6ff4a397043a48caa554a094d360fc1c, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.343719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2865767-6604-4afc-a2f4-09f635f74d14 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.355995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1040/2069] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:07 2022] PUT /identity/v3/projects/41a4d8fd02b2461a97110417ebf18745/users/6ff4a397043a48caa554a094d360fc1c/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:07.367377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff9e74-62b2-4e24-89ef-377d3f0fff80 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.367696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff9e74-62b2-4e24-89ef-377d3f0fff80 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.367974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff9e74-62b2-4e24-89ef-377d3f0fff80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.397510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e3ff9e74-62b2-4e24-89ef-377d3f0fff80 None None] MFA Rules not processed for user `6ff4a397043a48caa554a094d360fc1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:07.398759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e3ff9e74-62b2-4e24-89ef-377d3f0fff80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:07.466299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1030/2070] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:07.470778 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2865767-6604-4afc-a2f4-09f635f74d14 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.532428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2865767-6604-4afc-a2f4-09f635f74d14 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.536979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc3c06-a42a-4c04-a2e5-fb4c75f640d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.537564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc3c06-a42a-4c04-a2e5-fb4c75f640d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.537837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc3c06-a42a-4c04-a2e5-fb4c75f640d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.538103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc3c06-a42a-4c04-a2e5-fb4c75f640d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.538715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc3c06-a42a-4c04-a2e5-fb4c75f640d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.540643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc3c06-a42a-4c04-a2e5-fb4c75f640d9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.567983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1041/2071] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:15:07 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:07.572036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff9e74-62b2-4e24-89ef-377d3f0fff80 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.583118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff9e74-62b2-4e24-89ef-377d3f0fff80 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.640518 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ef1ea-617c-442d-94ae-c64455cf939e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.641039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ef1ea-617c-442d-94ae-c64455cf939e admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.641317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ef1ea-617c-442d-94ae-c64455cf939e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.641585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96ef1ea-617c-442d-94ae-c64455cf939e admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.642267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ef1ea-617c-442d-94ae-c64455cf939e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.644217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96ef1ea-617c-442d-94ae-c64455cf939e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.672123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1031/2072] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:15:07 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:07.673333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc3c06-a42a-4c04-a2e5-fb4c75f640d9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.683998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc3c06-a42a-4c04-a2e5-fb4c75f640d9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.689324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ff2fb-30bd-434c-907f-ab91288a6f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.689923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ff2fb-30bd-434c-907f-ab91288a6f94 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.690202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ff2fb-30bd-434c-907f-ab91288a6f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.690502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ff2fb-30bd-434c-907f-ab91288a6f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.691143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ff2fb-30bd-434c-907f-ab91288a6f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.692831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ff2fb-30bd-434c-907f-ab91288a6f94 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.706002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1042/2073] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:07.707293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ef1ea-617c-442d-94ae-c64455cf939e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.717734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96ef1ea-617c-442d-94ae-c64455cf939e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.722433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134b10d7-e86c-456c-8fbe-d7cc526b373b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.723033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b10d7-e86c-456c-8fbe-d7cc526b373b admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.723308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b10d7-e86c-456c-8fbe-d7cc526b373b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.723609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134b10d7-e86c-456c-8fbe-d7cc526b373b admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.724250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b10d7-e86c-456c-8fbe-d7cc526b373b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.725857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134b10d7-e86c-456c-8fbe-d7cc526b373b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.737789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1032/2074] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:07.742058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ff2fb-30bd-434c-907f-ab91288a6f94 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.753565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ff2fb-30bd-434c-907f-ab91288a6f94 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.758235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f185536-f808-4e03-95c5-b155e6884528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.758883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f185536-f808-4e03-95c5-b155e6884528 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.759158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f185536-f808-4e03-95c5-b155e6884528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.759430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f185536-f808-4e03-95c5-b155e6884528 admin admin] PATH_INFO: `/v3/projects/6304b660f3054ec5aca467f5b84a31fe/users/5c152c9ca86e4c79b20d225c6e613625/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.774267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f185536-f808-4e03-95c5-b155e6884528 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6304b660f3054ec5aca467f5b84a31fe, user_id=5c152c9ca86e4c79b20d225c6e613625, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.777004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f185536-f808-4e03-95c5-b155e6884528 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.789676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1043/2075] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:07 2022] PUT /identity/v3/projects/6304b660f3054ec5aca467f5b84a31fe/users/5c152c9ca86e4c79b20d225c6e613625/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:07.800769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e256-1338-4cac-9984-711f2e959c29 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.801056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e256-1338-4cac-9984-711f2e959c29 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.801332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e256-1338-4cac-9984-711f2e959c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.829302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-47b1e256-1338-4cac-9984-711f2e959c29 None None] MFA Rules not processed for user `5c152c9ca86e4c79b20d225c6e613625`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:07.830531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-47b1e256-1338-4cac-9984-711f2e959c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:07.858308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:07.896920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1033/2076] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:07.901558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f185536-f808-4e03-95c5-b155e6884528 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:07.921822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:07.964319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f185536-f808-4e03-95c5-b155e6884528 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:07.969626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1127a-03c1-4209-8e1e-4b49ba7c34df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:07.970253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1127a-03c1-4209-8e1e-4b49ba7c34df admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:07.970534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1127a-03c1-4209-8e1e-4b49ba7c34df admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:07.970806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1127a-03c1-4209-8e1e-4b49ba7c34df admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:07.971492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1127a-03c1-4209-8e1e-4b49ba7c34df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:07.973389 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1127a-03c1-4209-8e1e-4b49ba7c34df admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:07.999638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1044/2077] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:15:07 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:15:08.003814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1e256-1338-4cac-9984-711f2e959c29 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:08.015968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1e256-1338-4cac-9984-711f2e959c29 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:08.073613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf6ca4-c481-4547-9e57-e2af6b37da1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:08.074324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf6ca4-c481-4547-9e57-e2af6b37da1c admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:08.074605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf6ca4-c481-4547-9e57-e2af6b37da1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:08.074877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcf6ca4-c481-4547-9e57-e2af6b37da1c admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:08.075649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf6ca4-c481-4547-9e57-e2af6b37da1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:08.077568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcf6ca4-c481-4547-9e57-e2af6b37da1c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:08.108542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1034/2078] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:15:08 2022] POST /identity/v3/users => generated 295 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:08.112505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1127a-03c1-4209-8e1e-4b49ba7c34df admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:08.124059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1127a-03c1-4209-8e1e-4b49ba7c34df admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:08.128740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe224-3a2d-45b4-b289-744665e5729d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:08.129338 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe224-3a2d-45b4-b289-744665e5729d admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:08.129615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe224-3a2d-45b4-b289-744665e5729d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:08.129887 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ebe224-3a2d-45b4-b289-744665e5729d admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:08.130533 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebe224-3a2d-45b4-b289-744665e5729d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:08.132236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ebe224-3a2d-45b4-b289-744665e5729d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:08.143424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1045/2079] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:08.147915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf6ca4-c481-4547-9e57-e2af6b37da1c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:08.158733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcf6ca4-c481-4547-9e57-e2af6b37da1c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:08.163338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7efed8-b9fa-4d51-91c0-7b2bd764f48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:08.164059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7efed8-b9fa-4d51-91c0-7b2bd764f48c admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:08.164334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7efed8-b9fa-4d51-91c0-7b2bd764f48c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:08.164671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7efed8-b9fa-4d51-91c0-7b2bd764f48c admin admin] PATH_INFO: `/v3/projects/b36240c6cc8b4e6ba32b54005abac7f8/users/abda17d8c8fb4ffaba2510ab67a105cc/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:08.180190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7efed8-b9fa-4d51-91c0-7b2bd764f48c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36240c6cc8b4e6ba32b54005abac7f8, user_id=abda17d8c8fb4ffaba2510ab67a105cc, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:08.183169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7efed8-b9fa-4d51-91c0-7b2bd764f48c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:08.196302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1035/2080] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:08 2022] PUT /identity/v3/projects/b36240c6cc8b4e6ba32b54005abac7f8/users/abda17d8c8fb4ffaba2510ab67a105cc/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:08.200983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe224-3a2d-45b4-b289-744665e5729d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:08.210802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ebe224-3a2d-45b4-b289-744665e5729d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:08.267521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b9a3d-60dd-4df0-aa5d-b49e3921b03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:08.268115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b9a3d-60dd-4df0-aa5d-b49e3921b03c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:08.268387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b9a3d-60dd-4df0-aa5d-b49e3921b03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:08.268654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b9a3d-60dd-4df0-aa5d-b49e3921b03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:08.269286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b9a3d-60dd-4df0-aa5d-b49e3921b03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:08.270870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b9a3d-60dd-4df0-aa5d-b49e3921b03c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:08.281384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1046/2081] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:08.285501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7efed8-b9fa-4d51-91c0-7b2bd764f48c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:08.296751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7efed8-b9fa-4d51-91c0-7b2bd764f48c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:08.301236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2537b8-4597-46b4-8ce5-de06af30f8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:08.301874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2537b8-4597-46b4-8ce5-de06af30f8df admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:08.302179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2537b8-4597-46b4-8ce5-de06af30f8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:08.302450 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2537b8-4597-46b4-8ce5-de06af30f8df admin admin] PATH_INFO: `/v3/projects/b36240c6cc8b4e6ba32b54005abac7f8/users/abda17d8c8fb4ffaba2510ab67a105cc/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:08.304342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2537b8-4597-46b4-8ce5-de06af30f8df admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36240c6cc8b4e6ba32b54005abac7f8, user_id=abda17d8c8fb4ffaba2510ab67a105cc, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:08.306956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2537b8-4597-46b4-8ce5-de06af30f8df admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:08.318297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1036/2082] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:08 2022] PUT /identity/v3/projects/b36240c6cc8b4e6ba32b54005abac7f8/users/abda17d8c8fb4ffaba2510ab67a105cc/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:08.329684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57af7f-4480-4be0-bee4-f16897818922 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:08.329684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57af7f-4480-4be0-bee4-f16897818922 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:08.329885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e57af7f-4480-4be0-bee4-f16897818922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:08.360418 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4e57af7f-4480-4be0-bee4-f16897818922 None None] MFA Rules not processed for user `abda17d8c8fb4ffaba2510ab67a105cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:08.361627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4e57af7f-4480-4be0-bee4-f16897818922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:08.450704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1047/2083] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:08.462363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2537b8-4597-46b4-8ce5-de06af30f8df admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:08.464278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9c2537b8-4597-46b4-8ce5-de06af30f8df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:08.526443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2537b8-4597-46b4-8ce5-de06af30f8df admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:08.531106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d63bd-78f3-458c-9bc6-d7bea54b8124 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:08.531791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d63bd-78f3-458c-9bc6-d7bea54b8124 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:08.532224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d63bd-78f3-458c-9bc6-d7bea54b8124 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:08.532499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d63bd-78f3-458c-9bc6-d7bea54b8124 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:08.568229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:08.593739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d63bd-78f3-458c-9bc6-d7bea54b8124 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:08.595475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d63bd-78f3-458c-9bc6-d7bea54b8124 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:08.604255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1037/2084] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:11.175668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57af7f-4480-4be0-bee4-f16897818922 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:11.185290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e57af7f-4480-4be0-bee4-f16897818922 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:11.188829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27887293-38a7-4dc2-8a22-1b705e3361f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:11.189456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27887293-38a7-4dc2-8a22-1b705e3361f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:11.189739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27887293-38a7-4dc2-8a22-1b705e3361f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:11.189965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27887293-38a7-4dc2-8a22-1b705e3361f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:11.201060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:11.250589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27887293-38a7-4dc2-8a22-1b705e3361f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:11.252066 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27887293-38a7-4dc2-8a22-1b705e3361f1 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:11.260772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1048/2085] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:11.904236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d63bd-78f3-458c-9bc6-d7bea54b8124 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:11.912983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d63bd-78f3-458c-9bc6-d7bea54b8124 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:11.916711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11ab0f-a3cd-457c-8b76-f4854329a3c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:11.917300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ab0f-a3cd-457c-8b76-f4854329a3c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:11.917575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ab0f-a3cd-457c-8b76-f4854329a3c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:11.917840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11ab0f-a3cd-457c-8b76-f4854329a3c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:11.926713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11ab0f-a3cd-457c-8b76-f4854329a3c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:11.928167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11ab0f-a3cd-457c-8b76-f4854329a3c2 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:11.938000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1038/2086] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:11 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:15.852453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27887293-38a7-4dc2-8a22-1b705e3361f1 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:15.921125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27887293-38a7-4dc2-8a22-1b705e3361f1 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:15.925633 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:15.926267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:15.926544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:15.926813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] PATH_INFO: `/v3/users/6ff4a397043a48caa554a094d360fc1c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:15.927963 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ff4a397043a48caa554a094d360fc1c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:15.929947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:15.988463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ff4a397043a48caa554a094d360fc1c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.004201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ff4a397043a48caa554a094d360fc1c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.004832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ff4a397043a48caa554a094d360fc1c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.005424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ff4a397043a48caa554a094d360fc1c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.013290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ff4a397043a48caa554a094d360fc1c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.031536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1049/2087] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:15 2022] DELETE /identity/v3/users/6ff4a397043a48caa554a094d360fc1c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:16.036088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0a712-9d4a-4a46-b5d2-075a81a8365e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:16.036622 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0a712-9d4a-4a46-b5d2-075a81a8365e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:16.036736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c0a712-9d4a-4a46-b5d2-075a81a8365e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:16.056173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-11c0a712-9d4a-4a46-b5d2-075a81a8365e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:16.057481 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-11c0a712-9d4a-4a46-b5d2-075a81a8365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:16.119824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1039/2088] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:16.130220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:16.131665 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:16.168496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:16.212502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f59f00-333f-43a8-b707-09f1fee4bd01 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:16.216508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90846bf-f4c1-4496-babb-73234542b6cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:16.217128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90846bf-f4c1-4496-babb-73234542b6cf service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:16.217405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90846bf-f4c1-4496-babb-73234542b6cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:16.217677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90846bf-f4c1-4496-babb-73234542b6cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:16.227865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90846bf-f4c1-4496-babb-73234542b6cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:16.229335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90846bf-f4c1-4496-babb-73234542b6cf service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:16.240154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1050/2089] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:15:16.606715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0a712-9d4a-4a46-b5d2-075a81a8365e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:16.608443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-11c0a712-9d4a-4a46-b5d2-075a81a8365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:16.621249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c0a712-9d4a-4a46-b5d2-075a81a8365e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:16.626006 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:16.627310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:16.627621 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:16.627895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] PATH_INFO: `/v3/projects/41a4d8fd02b2461a97110417ebf18745` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:16.629031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] RBAC: Authorizing `identity:delete_project(project_id=41a4d8fd02b2461a97110417ebf18745)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:16.630977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:16.679534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41a4d8fd02b2461a97110417ebf18745', 'request_id': 'req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.686735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41a4d8fd02b2461a97110417ebf18745', 'request_id': 'req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.693059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41a4d8fd02b2461a97110417ebf18745', 'request_id': 'req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.703832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] The token cache is being invalidate because project 41a4d8fd02b2461a97110417ebf18745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:16.704115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.704746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.706328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1040/2090] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:16 2022] DELETE /identity/v3/projects/41a4d8fd02b2461a97110417ebf18745 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:16.710291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90846bf-f4c1-4496-babb-73234542b6cf service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:16.719972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90846bf-f4c1-4496-babb-73234542b6cf service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:16.782109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:16.783387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:16.783726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:16.784089 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] PATH_INFO: `/v3/users/5c152c9ca86e4c79b20d225c6e613625` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:16.785252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c152c9ca86e4c79b20d225c6e613625)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:16.787422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:16.836546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c152c9ca86e4c79b20d225c6e613625'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.844304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c152c9ca86e4c79b20d225c6e613625'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.844923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c152c9ca86e4c79b20d225c6e613625'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.846179 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c152c9ca86e4c79b20d225c6e613625'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.853287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c152c9ca86e4c79b20d225c6e613625'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:16.872164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1051/2091] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:16 2022] DELETE /identity/v3/users/5c152c9ca86e4c79b20d225c6e613625 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:17.217401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:17.219297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:17.240426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beefa8c3-a48f-493a-b2d5-7a168ea6a14e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:17.296656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:17.315888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:17.316526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:17.316802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:17.317071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] PATH_INFO: `/v3/projects/6304b660f3054ec5aca467f5b84a31fe` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:17.318187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6304b660f3054ec5aca467f5b84a31fe)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:17.320153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:17.365810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6304b660f3054ec5aca467f5b84a31fe', 'request_id': 'req-e3c54f33-f69e-464c-901c-e835f4e6b391'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:17.373361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6304b660f3054ec5aca467f5b84a31fe', 'request_id': 'req-e3c54f33-f69e-464c-901c-e835f4e6b391'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:17.380598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6304b660f3054ec5aca467f5b84a31fe', 'request_id': 'req-e3c54f33-f69e-464c-901c-e835f4e6b391'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:17.391704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] The token cache is being invalidate because project 6304b660f3054ec5aca467f5b84a31fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:17.391997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:17.392918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:17.394417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1041/2092] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:17 2022] DELETE /identity/v3/projects/6304b660f3054ec5aca467f5b84a31fe => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:17.398480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:17.399924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:17.412649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990663ac-13d3-4089-a7e8-bfd21671e01a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:17.463347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:17.474599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:17.475251 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:17.475854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:17.476175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] PATH_INFO: `/v3/users/abda17d8c8fb4ffaba2510ab67a105cc` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:17.477333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] RBAC: Authorizing `identity:delete_user(user_id=abda17d8c8fb4ffaba2510ab67a105cc)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:17.479301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:17.526667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abda17d8c8fb4ffaba2510ab67a105cc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:17.535233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abda17d8c8fb4ffaba2510ab67a105cc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:17.535868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abda17d8c8fb4ffaba2510ab67a105cc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:17.536462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abda17d8c8fb4ffaba2510ab67a105cc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:17.543391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b14ab5d1-a266-4799-9f5b-1a530851a19b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abda17d8c8fb4ffaba2510ab67a105cc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:17.560963 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1052/2093] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:17 2022] DELETE /identity/v3/users/abda17d8c8fb4ffaba2510ab67a105cc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:17.956308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:17.957759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:17.973348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c54f33-f69e-464c-901c-e835f4e6b391 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:18.031570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:18.032202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.032478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.033346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] PATH_INFO: `/v3/projects/b36240c6cc8b4e6ba32b54005abac7f8` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.034494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36240c6cc8b4e6ba32b54005abac7f8)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:18.036445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:18.084729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36240c6cc8b4e6ba32b54005abac7f8', 'request_id': 'req-de1dff10-2b65-4ab6-b0db-b3a76677952e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:18.091509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36240c6cc8b4e6ba32b54005abac7f8', 'request_id': 'req-de1dff10-2b65-4ab6-b0db-b3a76677952e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:18.097832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36240c6cc8b4e6ba32b54005abac7f8', 'request_id': 'req-de1dff10-2b65-4ab6-b0db-b3a76677952e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:18.109434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] The token cache is being invalidate because project b36240c6cc8b4e6ba32b54005abac7f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:18.109736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:18.110366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-de1dff10-2b65-4ab6-b0db-b3a76677952e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:18.112173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1042/2094] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:17 2022] DELETE /identity/v3/projects/b36240c6cc8b4e6ba32b54005abac7f8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:18.117034 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a39d0-7a27-47a6-ab11-1bec67c332c4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.117524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a39d0-7a27-47a6-ab11-1bec67c332c4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.117740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a39d0-7a27-47a6-ab11-1bec67c332c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.138547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-201a39d0-7a27-47a6-ab11-1bec67c332c4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:18.139786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-201a39d0-7a27-47a6-ab11-1bec67c332c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:18.203102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1053/2095] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:18.213102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcbc3c-9bf3-47f7-8c05-e62bd3d20675 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.213740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcbc3c-9bf3-47f7-8c05-e62bd3d20675 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.214029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcbc3c-9bf3-47f7-8c05-e62bd3d20675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.225281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:18.239016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a1fcbc3c-9bf3-47f7-8c05-e62bd3d20675 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:18.240354 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a1fcbc3c-9bf3-47f7-8c05-e62bd3d20675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:18.248661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1043/2096] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:18.253407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a39d0-7a27-47a6-ab11-1bec67c332c4 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:18.263261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a39d0-7a27-47a6-ab11-1bec67c332c4 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:18.272374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d78f8-dcd9-4477-a677-50969ce0987e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:18.272880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d78f8-dcd9-4477-a677-50969ce0987e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.273223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d78f8-dcd9-4477-a677-50969ce0987e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.273561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d78f8-dcd9-4477-a677-50969ce0987e admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.274346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d78f8-dcd9-4477-a677-50969ce0987e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:15:18.274711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d78f8-dcd9-4477-a677-50969ce0987e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:18.276422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d78f8-dcd9-4477-a677-50969ce0987e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:18.292606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1054/2097] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:15:18.293800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcbc3c-9bf3-47f7-8c05-e62bd3d20675 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:18.303108 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcbc3c-9bf3-47f7-8c05-e62bd3d20675 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:18.307807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d07426-6995-40e0-a58b-7893bf299879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:18.308433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d07426-6995-40e0-a58b-7893bf299879 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.308714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d07426-6995-40e0-a58b-7893bf299879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.309046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d07426-6995-40e0-a58b-7893bf299879 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.309683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d07426-6995-40e0-a58b-7893bf299879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:18.311616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d07426-6995-40e0-a58b-7893bf299879 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:18.340669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1044/2098] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:15:18 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:18.346879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d78f8-dcd9-4477-a677-50969ce0987e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:18.357509 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d78f8-dcd9-4477-a677-50969ce0987e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:18.418882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23ea1b-c0dc-42ac-aa63-a76384274a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:18.419659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23ea1b-c0dc-42ac-aa63-a76384274a9f admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.419944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23ea1b-c0dc-42ac-aa63-a76384274a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.420222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d23ea1b-c0dc-42ac-aa63-a76384274a9f admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.420873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23ea1b-c0dc-42ac-aa63-a76384274a9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:18.422833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d23ea1b-c0dc-42ac-aa63-a76384274a9f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:18.432089 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:18.453930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1055/2099] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:15:18 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:18.455258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d07426-6995-40e0-a58b-7893bf299879 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:18.466213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d07426-6995-40e0-a58b-7893bf299879 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:18.471856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f9c48-a260-412d-98de-9c78e93181f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:18.472492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f9c48-a260-412d-98de-9c78e93181f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.472885 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f9c48-a260-412d-98de-9c78e93181f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.473166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141f9c48-a260-412d-98de-9c78e93181f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.473841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f9c48-a260-412d-98de-9c78e93181f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:18.475565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141f9c48-a260-412d-98de-9c78e93181f2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:18.487294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1045/2100] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:15:18.491513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23ea1b-c0dc-42ac-aa63-a76384274a9f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:18.505778 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d23ea1b-c0dc-42ac-aa63-a76384274a9f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:18.510662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907dea7e-f83e-4ced-94c3-1131b6fa108b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:18.511258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dea7e-f83e-4ced-94c3-1131b6fa108b admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.511567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dea7e-f83e-4ced-94c3-1131b6fa108b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.511839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907dea7e-f83e-4ced-94c3-1131b6fa108b admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.512482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907dea7e-f83e-4ced-94c3-1131b6fa108b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:18.514087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907dea7e-f83e-4ced-94c3-1131b6fa108b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:18.524947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1056/2101] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:15:18.529175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f9c48-a260-412d-98de-9c78e93181f2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:18.541099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141f9c48-a260-412d-98de-9c78e93181f2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:18.547169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c921f-6e82-4a44-bf68-744c76c5ba25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:18.548261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c921f-6e82-4a44-bf68-744c76c5ba25 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.549265 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c921f-6e82-4a44-bf68-744c76c5ba25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.549704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c921f-6e82-4a44-bf68-744c76c5ba25 admin admin] PATH_INFO: `/v3/projects/d14cf31db1244e5d9f5330a55f6dc801/users/bae66efbdf994be2b6f49ba3e5c08ab6/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.566280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c921f-6e82-4a44-bf68-744c76c5ba25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d14cf31db1244e5d9f5330a55f6dc801, user_id=bae66efbdf994be2b6f49ba3e5c08ab6, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:18.569076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c921f-6e82-4a44-bf68-744c76c5ba25 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:18.583771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1046/2102] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:18 2022] PUT /identity/v3/projects/d14cf31db1244e5d9f5330a55f6dc801/users/bae66efbdf994be2b6f49ba3e5c08ab6/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:18.589180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30184057-54fe-42af-b30a-ef3e9339c7e4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.589513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30184057-54fe-42af-b30a-ef3e9339c7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.589790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30184057-54fe-42af-b30a-ef3e9339c7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.609760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-30184057-54fe-42af-b30a-ef3e9339c7e4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:18.611487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-30184057-54fe-42af-b30a-ef3e9339c7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:18.670068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1057/2103] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:18.680245 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c921f-6e82-4a44-bf68-744c76c5ba25 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:18.681915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d73c921f-6e82-4a44-bf68-744c76c5ba25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:18.742012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c921f-6e82-4a44-bf68-744c76c5ba25 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:18.745569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06307af-d303-4a24-95a3-3ba9ab931f4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:18.746172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06307af-d303-4a24-95a3-3ba9ab931f4c service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:18.746477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06307af-d303-4a24-95a3-3ba9ab931f4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:18.746746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06307af-d303-4a24-95a3-3ba9ab931f4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:18.756356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06307af-d303-4a24-95a3-3ba9ab931f4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:18.757781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06307af-d303-4a24-95a3-3ba9ab931f4c service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:18.767004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1047/2104] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:19.501357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba04dd0-7fe4-4cbf-a13e-2f341e29eba4 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:19.502424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba04dd0-7fe4-4cbf-a13e-2f341e29eba4 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:19.502777 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba04dd0-7fe4-4cbf-a13e-2f341e29eba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:19.533469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fba04dd0-7fe4-4cbf-a13e-2f341e29eba4 None None] MFA Rules not processed for user `bae66efbdf994be2b6f49ba3e5c08ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:19.534837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fba04dd0-7fe4-4cbf-a13e-2f341e29eba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:19.621046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1058/2105] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:19.632043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06307af-d303-4a24-95a3-3ba9ab931f4c service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:19.633684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c06307af-d303-4a24-95a3-3ba9ab931f4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:19.717915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06307af-d303-4a24-95a3-3ba9ab931f4c service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:19.721995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6729399-1669-4593-b27c-1900a408f5d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:19.722828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6729399-1669-4593-b27c-1900a408f5d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:19.723264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6729399-1669-4593-b27c-1900a408f5d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:19.723714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6729399-1669-4593-b27c-1900a408f5d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:19.733331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6729399-1669-4593-b27c-1900a408f5d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:19.734945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6729399-1669-4593-b27c-1900a408f5d9 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:19.745981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1048/2106] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:15:21.150211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9016f3-f65a-4284-8676-e376c5f3184a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:21.150789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9016f3-f65a-4284-8676-e376c5f3184a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:21.150789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9016f3-f65a-4284-8676-e376c5f3184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:21.172603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bf9016f3-f65a-4284-8676-e376c5f3184a None None] MFA Rules not processed for user `bae66efbdf994be2b6f49ba3e5c08ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:21.174412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bf9016f3-f65a-4284-8676-e376c5f3184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:21.182341 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1059/2107] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:23.984759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6729399-1669-4593-b27c-1900a408f5d9 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:23.986267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b6729399-1669-4593-b27c-1900a408f5d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:23.999937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6729399-1669-4593-b27c-1900a408f5d9 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:24.004418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:24.005029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.005303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.005568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] PATH_INFO: `/v3/users/5ca58aca3f9d4d8fa9f1c6315db7f54d` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.006642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ca58aca3f9d4d8fa9f1c6315db7f54d)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:24.008568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:24.067100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:24.073571 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ca58aca3f9d4d8fa9f1c6315db7f54d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:24.074622 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ca58aca3f9d4d8fa9f1c6315db7f54d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:24.086469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ca58aca3f9d4d8fa9f1c6315db7f54d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:24.096576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ca58aca3f9d4d8fa9f1c6315db7f54d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:24.097280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3d36533f-d5b7-431b-95d2-d76981e5e022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ca58aca3f9d4d8fa9f1c6315db7f54d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:24.118262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1049/2108] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:23 2022] DELETE /identity/v3/users/5ca58aca3f9d4d8fa9f1c6315db7f54d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:24.324361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9016f3-f65a-4284-8676-e376c5f3184a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:24.341601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9016f3-f65a-4284-8676-e376c5f3184a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:24.412045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:24.412693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.412938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.413881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] PATH_INFO: `/v3/projects/06101fb7de2e4338967b29365e713677` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.414551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06101fb7de2e4338967b29365e713677)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:24.416768 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:24.419707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:24.481275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06101fb7de2e4338967b29365e713677', 'request_id': 'req-f7ce4633-75c7-4a55-a609-5c92472d88a5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:24.487835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06101fb7de2e4338967b29365e713677', 'request_id': 'req-f7ce4633-75c7-4a55-a609-5c92472d88a5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:24.499972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06101fb7de2e4338967b29365e713677', 'request_id': 'req-f7ce4633-75c7-4a55-a609-5c92472d88a5'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:24.513010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] The token cache is being invalidate because project 06101fb7de2e4338967b29365e713677 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:24.513296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:24.513960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f7ce4633-75c7-4a55-a609-5c92472d88a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:24.515430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1060/2109] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:24 2022] DELETE /identity/v3/projects/06101fb7de2e4338967b29365e713677 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:15:24.521357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0a9af-0a2d-476c-bd3c-e03acc7b3e49 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.521722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0a9af-0a2d-476c-bd3c-e03acc7b3e49 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.522067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb0a9af-0a2d-476c-bd3c-e03acc7b3e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.542600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-8cb0a9af-0a2d-476c-bd3c-e03acc7b3e49 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:24.544047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-8cb0a9af-0a2d-476c-bd3c-e03acc7b3e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:24.609561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1050/2110] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:24.618081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311307b5-c985-4436-b930-8b576550145e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.618440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311307b5-c985-4436-b930-8b576550145e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.618781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311307b5-c985-4436-b930-8b576550145e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.639826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-311307b5-c985-4436-b930-8b576550145e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:24.641258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-311307b5-c985-4436-b930-8b576550145e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:24.649476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1061/2111] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:15:24.653719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0a9af-0a2d-476c-bd3c-e03acc7b3e49 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:24.664158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb0a9af-0a2d-476c-bd3c-e03acc7b3e49 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:24.668780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50c7a8-e70c-4726-a3ad-304841030641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:24.669408 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50c7a8-e70c-4726-a3ad-304841030641 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.669717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50c7a8-e70c-4726-a3ad-304841030641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.669949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50c7a8-e70c-4726-a3ad-304841030641 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.670934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50c7a8-e70c-4726-a3ad-304841030641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:15:24.670934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50c7a8-e70c-4726-a3ad-304841030641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:24.672519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50c7a8-e70c-4726-a3ad-304841030641 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:24.686139 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1051/2112] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:15:24.691091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311307b5-c985-4436-b930-8b576550145e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:24.700579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311307b5-c985-4436-b930-8b576550145e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:24.705537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81902818-ed57-49cc-95e3-de766df7d590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:24.706202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81902818-ed57-49cc-95e3-de766df7d590 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.706487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81902818-ed57-49cc-95e3-de766df7d590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.706758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81902818-ed57-49cc-95e3-de766df7d590 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.707414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81902818-ed57-49cc-95e3-de766df7d590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:24.709388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81902818-ed57-49cc-95e3-de766df7d590 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:24.740323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1062/2113] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:15:24 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:24.741427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50c7a8-e70c-4726-a3ad-304841030641 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:24.752040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50c7a8-e70c-4726-a3ad-304841030641 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:24.809526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ed410-280f-460c-909f-3d9628de6b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:24.810125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ed410-280f-460c-909f-3d9628de6b1c admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.810397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ed410-280f-460c-909f-3d9628de6b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.810662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72ed410-280f-460c-909f-3d9628de6b1c admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.811291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ed410-280f-460c-909f-3d9628de6b1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:24.813320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72ed410-280f-460c-909f-3d9628de6b1c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:24.838190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1052/2114] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:15:24 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:15:24.839392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81902818-ed57-49cc-95e3-de766df7d590 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:24.849551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81902818-ed57-49cc-95e3-de766df7d590 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:24.854228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309dd4-f939-473c-ae43-b2983148158c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:24.854943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309dd4-f939-473c-ae43-b2983148158c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.855230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309dd4-f939-473c-ae43-b2983148158c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.855530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07309dd4-f939-473c-ae43-b2983148158c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.856185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07309dd4-f939-473c-ae43-b2983148158c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:24.858165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07309dd4-f939-473c-ae43-b2983148158c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:24.869837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1063/2115] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:15:24.873727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ed410-280f-460c-909f-3d9628de6b1c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:24.884974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72ed410-280f-460c-909f-3d9628de6b1c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:24.889633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae2d85-4282-4cb3-8cb4-c9f78c22ca0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:24.890232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae2d85-4282-4cb3-8cb4-c9f78c22ca0f admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.890507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae2d85-4282-4cb3-8cb4-c9f78c22ca0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.890771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae2d85-4282-4cb3-8cb4-c9f78c22ca0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.891403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae2d85-4282-4cb3-8cb4-c9f78c22ca0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:24.893447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae2d85-4282-4cb3-8cb4-c9f78c22ca0f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:24.903993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1053/2116] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:24.908365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309dd4-f939-473c-ae43-b2983148158c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:24.918426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07309dd4-f939-473c-ae43-b2983148158c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:24.923026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168ba81-bac2-41cc-b36f-293ff474bf3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:24.924252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168ba81-bac2-41cc-b36f-293ff474bf3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.924524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168ba81-bac2-41cc-b36f-293ff474bf3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.924797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168ba81-bac2-41cc-b36f-293ff474bf3e admin admin] PATH_INFO: `/v3/projects/a5f4551cf6234a23a4b291a7a33c9763/users/dce6e551c35e41bea3011d6aa2163931/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.940310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168ba81-bac2-41cc-b36f-293ff474bf3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f4551cf6234a23a4b291a7a33c9763, user_id=dce6e551c35e41bea3011d6aa2163931, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:24.943114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168ba81-bac2-41cc-b36f-293ff474bf3e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:24.958716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1064/2117] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:24 2022] PUT /identity/v3/projects/a5f4551cf6234a23a4b291a7a33c9763/users/dce6e551c35e41bea3011d6aa2163931/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:24.968660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5818ffd-c2fd-432b-a5ff-4e4d41d44497 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:24.968990 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5818ffd-c2fd-432b-a5ff-4e4d41d44497 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:24.969274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5818ffd-c2fd-432b-a5ff-4e4d41d44497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:24.999301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d5818ffd-c2fd-432b-a5ff-4e4d41d44497 None None] MFA Rules not processed for user `dce6e551c35e41bea3011d6aa2163931`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:25.000532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d5818ffd-c2fd-432b-a5ff-4e4d41d44497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:25.013524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:25.069780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1054/2118] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:25.074213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168ba81-bac2-41cc-b36f-293ff474bf3e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:25.137504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168ba81-bac2-41cc-b36f-293ff474bf3e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:25.142245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4e2e4-185a-4ea5-adf8-9cf184c923e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:25.142843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4e2e4-185a-4ea5-adf8-9cf184c923e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:25.143121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4e2e4-185a-4ea5-adf8-9cf184c923e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:25.143396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c4e2e4-185a-4ea5-adf8-9cf184c923e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:25.144065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4e2e4-185a-4ea5-adf8-9cf184c923e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:25.145996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c4e2e4-185a-4ea5-adf8-9cf184c923e6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:25.185185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1065/2119] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:15:25 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:25.188718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5818ffd-c2fd-432b-a5ff-4e4d41d44497 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:25.201887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5818ffd-c2fd-432b-a5ff-4e4d41d44497 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:25.274459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0a286-715c-4a41-ab20-250c679fa429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:25.275374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0a286-715c-4a41-ab20-250c679fa429 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:25.275698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0a286-715c-4a41-ab20-250c679fa429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:25.275978 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0a286-715c-4a41-ab20-250c679fa429 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:25.276644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0a286-715c-4a41-ab20-250c679fa429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:25.278636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0a286-715c-4a41-ab20-250c679fa429 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:25.316384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1055/2120] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:15:25 2022] POST /identity/v3/users => generated 295 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:15:25.320175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4e2e4-185a-4ea5-adf8-9cf184c923e6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:25.343106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c4e2e4-185a-4ea5-adf8-9cf184c923e6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:25.354330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7f74f-79ff-406b-967f-90adff7c9404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:25.354974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7f74f-79ff-406b-967f-90adff7c9404 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:25.355332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7f74f-79ff-406b-967f-90adff7c9404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:25.355824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7f74f-79ff-406b-967f-90adff7c9404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:25.356793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7f74f-79ff-406b-967f-90adff7c9404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:25.358739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7f74f-79ff-406b-967f-90adff7c9404 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:25.370236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1066/2121] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:25.374681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0a286-715c-4a41-ab20-250c679fa429 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:25.384787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0a286-715c-4a41-ab20-250c679fa429 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:25.389344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa53f91-bbbc-4b82-8b5d-93562cada955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:25.389994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa53f91-bbbc-4b82-8b5d-93562cada955 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:25.390268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa53f91-bbbc-4b82-8b5d-93562cada955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:25.390540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa53f91-bbbc-4b82-8b5d-93562cada955 admin admin] PATH_INFO: `/v3/projects/16df779c57884ef2878acd1d43846a95/users/40fd729123594508bac4afd50e1996a8/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:25.406544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa53f91-bbbc-4b82-8b5d-93562cada955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16df779c57884ef2878acd1d43846a95, user_id=40fd729123594508bac4afd50e1996a8, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:25.410378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa53f91-bbbc-4b82-8b5d-93562cada955 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:25.424207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1056/2122] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:25 2022] PUT /identity/v3/projects/16df779c57884ef2878acd1d43846a95/users/40fd729123594508bac4afd50e1996a8/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:25.427905 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7f74f-79ff-406b-967f-90adff7c9404 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:25.433072 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:25.440055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7f74f-79ff-406b-967f-90adff7c9404 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:25.500544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de833c5-7978-4cbf-8132-57f394cf5d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:25.501424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de833c5-7978-4cbf-8132-57f394cf5d86 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:25.501710 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de833c5-7978-4cbf-8132-57f394cf5d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:25.501962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de833c5-7978-4cbf-8132-57f394cf5d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:25.504523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de833c5-7978-4cbf-8132-57f394cf5d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:25.506311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de833c5-7978-4cbf-8132-57f394cf5d86 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:25.518393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1067/2123] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:25 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:25.524039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa53f91-bbbc-4b82-8b5d-93562cada955 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:25.537110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa53f91-bbbc-4b82-8b5d-93562cada955 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:25.541900 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49345dc5-8dcb-4f04-8552-581c054f831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:25.542684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49345dc5-8dcb-4f04-8552-581c054f831c admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:25.543034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49345dc5-8dcb-4f04-8552-581c054f831c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:25.543372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49345dc5-8dcb-4f04-8552-581c054f831c admin admin] PATH_INFO: `/v3/projects/16df779c57884ef2878acd1d43846a95/users/40fd729123594508bac4afd50e1996a8/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:25.545359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49345dc5-8dcb-4f04-8552-581c054f831c admin admin] RBAC: Authorizing `identity:create_grant(project_id=16df779c57884ef2878acd1d43846a95, user_id=40fd729123594508bac4afd50e1996a8, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:25.548747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49345dc5-8dcb-4f04-8552-581c054f831c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:25.559956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1057/2124] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:25 2022] PUT /identity/v3/projects/16df779c57884ef2878acd1d43846a95/users/40fd729123594508bac4afd50e1996a8/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:15:25.573575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0d163-f914-43cd-a2ec-0088e4a156a0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:25.573881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0d163-f914-43cd-a2ec-0088e4a156a0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:25.574148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a0d163-f914-43cd-a2ec-0088e4a156a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:25.603942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-82a0d163-f914-43cd-a2ec-0088e4a156a0 None None] MFA Rules not processed for user `40fd729123594508bac4afd50e1996a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:25.605164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-82a0d163-f914-43cd-a2ec-0088e4a156a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:25.678274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1068/2125] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:25.690406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49345dc5-8dcb-4f04-8552-581c054f831c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:25.692367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-49345dc5-8dcb-4f04-8552-581c054f831c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:25.753829 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49345dc5-8dcb-4f04-8552-581c054f831c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:25.761941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0a052-3ad4-4751-83c3-824fc4878880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:25.762550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0a052-3ad4-4751-83c3-824fc4878880 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:25.762823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0a052-3ad4-4751-83c3-824fc4878880 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:25.763090 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0a052-3ad4-4751-83c3-824fc4878880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:25.773920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0a052-3ad4-4751-83c3-824fc4878880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:25.775402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0a052-3ad4-4751-83c3-824fc4878880 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:25.785751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1058/2126] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:25.869299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0d163-f914-43cd-a2ec-0088e4a156a0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:25.878817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a0d163-f914-43cd-a2ec-0088e4a156a0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:25.882400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba06923c-31fd-43d2-9f86-14c7d9a66f38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:25.883001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06923c-31fd-43d2-9f86-14c7d9a66f38 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:25.883278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06923c-31fd-43d2-9f86-14c7d9a66f38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:25.883644 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba06923c-31fd-43d2-9f86-14c7d9a66f38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:25.940891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06923c-31fd-43d2-9f86-14c7d9a66f38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:25.942281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba06923c-31fd-43d2-9f86-14c7d9a66f38 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:25.951231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1069/2127] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:35.780956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=91,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:35.947098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:38.960138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0a052-3ad4-4751-83c3-824fc4878880 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:39.033006 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0a052-3ad4-4751-83c3-824fc4878880 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:39.038392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:39.039151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:39.039540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:39.039818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] PATH_INFO: `/v3/users/dce6e551c35e41bea3011d6aa2163931` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:39.040969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce6e551c35e41bea3011d6aa2163931)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:39.042945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:39.104326 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce6e551c35e41bea3011d6aa2163931'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.104885 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce6e551c35e41bea3011d6aa2163931'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.113355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce6e551c35e41bea3011d6aa2163931'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.120487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce6e551c35e41bea3011d6aa2163931'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.121159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce6e551c35e41bea3011d6aa2163931'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.139184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1059/2128] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:38 2022] DELETE /identity/v3/users/dce6e551c35e41bea3011d6aa2163931 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:39.143649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62dd2b-0982-4dfa-a1d6-7568e09e4e3a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:39.144147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62dd2b-0982-4dfa-a1d6-7568e09e4e3a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:39.144227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba62dd2b-0982-4dfa-a1d6-7568e09e4e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:39.166257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ba62dd2b-0982-4dfa-a1d6-7568e09e4e3a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:39.167490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ba62dd2b-0982-4dfa-a1d6-7568e09e4e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:39.239200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1070/2129] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:39.245546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:39.246958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:39.305233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87261f7f-a595-409d-8521-b1e30238066e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:39.308858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c65189-edaa-44f1-afc4-e1cebc269262 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:39.309461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c65189-edaa-44f1-afc4-e1cebc269262 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:39.309737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c65189-edaa-44f1-afc4-e1cebc269262 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:39.310006 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c65189-edaa-44f1-afc4-e1cebc269262 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:39.319610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c65189-edaa-44f1-afc4-e1cebc269262 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:39.321047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c65189-edaa-44f1-afc4-e1cebc269262 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:39.331218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1060/2130] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:39.606931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62dd2b-0982-4dfa-a1d6-7568e09e4e3a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:39.608666 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ba62dd2b-0982-4dfa-a1d6-7568e09e4e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:39.621944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba62dd2b-0982-4dfa-a1d6-7568e09e4e3a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:39.627780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:39.627928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:39.628218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:39.628499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] PATH_INFO: `/v3/projects/a5f4551cf6234a23a4b291a7a33c9763` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:39.629637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f4551cf6234a23a4b291a7a33c9763)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:39.631606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:39.679192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f4551cf6234a23a4b291a7a33c9763', 'request_id': 'req-fab5ad47-20b2-4116-a05b-111f008f5197'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.684727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f4551cf6234a23a4b291a7a33c9763', 'request_id': 'req-fab5ad47-20b2-4116-a05b-111f008f5197'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.692026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f4551cf6234a23a4b291a7a33c9763', 'request_id': 'req-fab5ad47-20b2-4116-a05b-111f008f5197'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.702869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] The token cache is being invalidate because project a5f4551cf6234a23a4b291a7a33c9763 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:39.703153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.703803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.705873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1071/2131] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:39 2022] DELETE /identity/v3/projects/a5f4551cf6234a23a4b291a7a33c9763 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:39.709904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c65189-edaa-44f1-afc4-e1cebc269262 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:39.719575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c65189-edaa-44f1-afc4-e1cebc269262 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:39.741942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:39.781966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:39.783318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:39.783636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:39.783917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] PATH_INFO: `/v3/users/40fd729123594508bac4afd50e1996a8` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:39.785113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40fd729123594508bac4afd50e1996a8)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:39.787035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:39.831126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40fd729123594508bac4afd50e1996a8'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.831864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40fd729123594508bac4afd50e1996a8'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.838695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40fd729123594508bac4afd50e1996a8'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.846138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40fd729123594508bac4afd50e1996a8'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.846774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9cbbc212-8f95-41e9-964b-dfade0372630 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40fd729123594508bac4afd50e1996a8'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:39.864138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1061/2132] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:39 2022] DELETE /identity/v3/users/40fd729123594508bac4afd50e1996a8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:40.178968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:40.180451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:40.194134 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab5ad47-20b2-4116-a05b-111f008f5197 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:40.250345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:40.251588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.251866 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.252131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] PATH_INFO: `/v3/projects/16df779c57884ef2878acd1d43846a95` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.253223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16df779c57884ef2878acd1d43846a95)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:40.255115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:40.285452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:40.302692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16df779c57884ef2878acd1d43846a95', 'request_id': 'req-1d6277c9-53f5-41d4-9cce-5b43da0fff76'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:40.308115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16df779c57884ef2878acd1d43846a95', 'request_id': 'req-1d6277c9-53f5-41d4-9cce-5b43da0fff76'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:40.316278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16df779c57884ef2878acd1d43846a95', 'request_id': 'req-1d6277c9-53f5-41d4-9cce-5b43da0fff76'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:40.327758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] The token cache is being invalidate because project 16df779c57884ef2878acd1d43846a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:40.328207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:40.328994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1d6277c9-53f5-41d4-9cce-5b43da0fff76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:40.331704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1072/2133] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:40 2022] DELETE /identity/v3/projects/16df779c57884ef2878acd1d43846a95 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:40.336435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662a2b9-5a0e-4a52-8fd5-00446f2224cb None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.336877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662a2b9-5a0e-4a52-8fd5-00446f2224cb None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.336974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662a2b9-5a0e-4a52-8fd5-00446f2224cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.356285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-5662a2b9-5a0e-4a52-8fd5-00446f2224cb None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:40.357733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5662a2b9-5a0e-4a52-8fd5-00446f2224cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:40.415863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1062/2134] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:40.424462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f7fff-a5e2-42c8-89ab-2807457cd31e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.424679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f7fff-a5e2-42c8-89ab-2807457cd31e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.424984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4f7fff-a5e2-42c8-89ab-2807457cd31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.447076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2f4f7fff-a5e2-42c8-89ab-2807457cd31e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:40.449243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2f4f7fff-a5e2-42c8-89ab-2807457cd31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:40.460446 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1073/2135] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:40.464624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662a2b9-5a0e-4a52-8fd5-00446f2224cb None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:40.474916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662a2b9-5a0e-4a52-8fd5-00446f2224cb None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:40.479475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0e74e-28dc-4172-a181-69f97aee2c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:40.480081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0e74e-28dc-4172-a181-69f97aee2c19 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.480354 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0e74e-28dc-4172-a181-69f97aee2c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.480621 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0e74e-28dc-4172-a181-69f97aee2c19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.481261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0e74e-28dc-4172-a181-69f97aee2c19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:15:40.481692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0e74e-28dc-4172-a181-69f97aee2c19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:40.483630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0e74e-28dc-4172-a181-69f97aee2c19 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:40.495976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1063/2136] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:15:40.500389 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f7fff-a5e2-42c8-89ab-2807457cd31e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:40.510342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4f7fff-a5e2-42c8-89ab-2807457cd31e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:40.514980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fef47-be64-4e47-b4b5-cd591d2884b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:40.515619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fef47-be64-4e47-b4b5-cd591d2884b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.515900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fef47-be64-4e47-b4b5-cd591d2884b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.516173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4fef47-be64-4e47-b4b5-cd591d2884b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.517391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fef47-be64-4e47-b4b5-cd591d2884b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:40.519278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4fef47-be64-4e47-b4b5-cd591d2884b4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:40.549355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1074/2137] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:15:40 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:40.553615 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0e74e-28dc-4172-a181-69f97aee2c19 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:40.563844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0e74e-28dc-4172-a181-69f97aee2c19 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:40.592518 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:40.626781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c6e64-a279-4aae-be47-e79c9e8f7e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:40.627503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c6e64-a279-4aae-be47-e79c9e8f7e74 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.627849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c6e64-a279-4aae-be47-e79c9e8f7e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.628156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c6e64-a279-4aae-be47-e79c9e8f7e74 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.628786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c6e64-a279-4aae-be47-e79c9e8f7e74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:40.630696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6c6e64-a279-4aae-be47-e79c9e8f7e74 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:40.652325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1064/2138] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:15:40 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:40.656400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fef47-be64-4e47-b4b5-cd591d2884b4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:40.667112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4fef47-be64-4e47-b4b5-cd591d2884b4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:40.671771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f8e36-7906-44e5-91b7-a09351d8ded2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:40.672359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f8e36-7906-44e5-91b7-a09351d8ded2 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.672635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f8e36-7906-44e5-91b7-a09351d8ded2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.672978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016f8e36-7906-44e5-91b7-a09351d8ded2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.673756 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f8e36-7906-44e5-91b7-a09351d8ded2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:40.675505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016f8e36-7906-44e5-91b7-a09351d8ded2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:40.685857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1075/2139] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:40.690346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c6e64-a279-4aae-be47-e79c9e8f7e74 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:40.700196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c6e64-a279-4aae-be47-e79c9e8f7e74 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:40.705636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963c9066-1935-4e10-8392-ccd5be6de86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:40.706229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c9066-1935-4e10-8392-ccd5be6de86b admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.706508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c9066-1935-4e10-8392-ccd5be6de86b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.706777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963c9066-1935-4e10-8392-ccd5be6de86b admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.707409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c9066-1935-4e10-8392-ccd5be6de86b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:40.709036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963c9066-1935-4e10-8392-ccd5be6de86b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:40.719364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1065/2140] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:15:40.723805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f8e36-7906-44e5-91b7-a09351d8ded2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:40.734760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016f8e36-7906-44e5-91b7-a09351d8ded2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:40.739408 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d498a79-9631-4b6d-b4b1-3f08bf2fca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:40.740104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d498a79-9631-4b6d-b4b1-3f08bf2fca05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.740391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d498a79-9631-4b6d-b4b1-3f08bf2fca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.740665 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d498a79-9631-4b6d-b4b1-3f08bf2fca05 admin admin] PATH_INFO: `/v3/projects/6d59b0d6f1dd4974be3f7c0e6349bb57/users/5292f230d5924eaf91f5a6ab0887e78a/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.756287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d498a79-9631-4b6d-b4b1-3f08bf2fca05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d59b0d6f1dd4974be3f7c0e6349bb57, user_id=5292f230d5924eaf91f5a6ab0887e78a, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:40.759038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d498a79-9631-4b6d-b4b1-3f08bf2fca05 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:40.772636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1076/2141] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:40 2022] PUT /identity/v3/projects/6d59b0d6f1dd4974be3f7c0e6349bb57/users/5292f230d5924eaf91f5a6ab0887e78a/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:15:40.784063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695f314-1ec0-4e0b-adb3-c6370533ac76 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.784405 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695f314-1ec0-4e0b-adb3-c6370533ac76 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.784624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7695f314-1ec0-4e0b-adb3-c6370533ac76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.815749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7695f314-1ec0-4e0b-adb3-c6370533ac76 None None] MFA Rules not processed for user `5292f230d5924eaf91f5a6ab0887e78a`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:40.817013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7695f314-1ec0-4e0b-adb3-c6370533ac76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:40.890440 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1066/2142] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:15:40.895942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d498a79-9631-4b6d-b4b1-3f08bf2fca05 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:40.960608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d498a79-9631-4b6d-b4b1-3f08bf2fca05 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:40.965238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28afd7-88ac-451f-b080-6b06cd7ddc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:40.965834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28afd7-88ac-451f-b080-6b06cd7ddc9c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:40.966111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28afd7-88ac-451f-b080-6b06cd7ddc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:40.966382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28afd7-88ac-451f-b080-6b06cd7ddc9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:40.967000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28afd7-88ac-451f-b080-6b06cd7ddc9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:40.968968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28afd7-88ac-451f-b080-6b06cd7ddc9c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:40.996161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1077/2143] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:15:40 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:41.000307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695f314-1ec0-4e0b-adb3-c6370533ac76 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:41.011182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7695f314-1ec0-4e0b-adb3-c6370533ac76 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:41.068054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ff838-5d18-4c6d-88f7-fabd7bd88e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:41.068647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff838-5d18-4c6d-88f7-fabd7bd88e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:41.068927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff838-5d18-4c6d-88f7-fabd7bd88e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:41.069198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9ff838-5d18-4c6d-88f7-fabd7bd88e4b admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:41.069830 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ff838-5d18-4c6d-88f7-fabd7bd88e4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:41.071833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9ff838-5d18-4c6d-88f7-fabd7bd88e4b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:41.092973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1067/2144] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:15:40 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:15:41.097604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28afd7-88ac-451f-b080-6b06cd7ddc9c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:41.108172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28afd7-88ac-451f-b080-6b06cd7ddc9c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:41.112939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25210f-5279-414f-9611-558910bfab45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:41.113528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25210f-5279-414f-9611-558910bfab45 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:41.113803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25210f-5279-414f-9611-558910bfab45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:41.114224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac25210f-5279-414f-9611-558910bfab45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:41.114866 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25210f-5279-414f-9611-558910bfab45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:41.116511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac25210f-5279-414f-9611-558910bfab45 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:41.127246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1078/2145] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:41.131357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ff838-5d18-4c6d-88f7-fabd7bd88e4b admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:41.141302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9ff838-5d18-4c6d-88f7-fabd7bd88e4b admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:41.146519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545e5a7-b4e3-4be6-ad60-bf523566dd30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:41.147171 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545e5a7-b4e3-4be6-ad60-bf523566dd30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:41.147463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545e5a7-b4e3-4be6-ad60-bf523566dd30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:41.147777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d545e5a7-b4e3-4be6-ad60-bf523566dd30 admin admin] PATH_INFO: `/v3/projects/f90ae5d820bb4459b2934efbf3a007c9/users/25b5239a5bbe42bb9d829e1d552b307d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:41.163020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545e5a7-b4e3-4be6-ad60-bf523566dd30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90ae5d820bb4459b2934efbf3a007c9, user_id=25b5239a5bbe42bb9d829e1d552b307d, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:41.165730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d545e5a7-b4e3-4be6-ad60-bf523566dd30 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:41.180240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1068/2146] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:41 2022] PUT /identity/v3/projects/f90ae5d820bb4459b2934efbf3a007c9/users/25b5239a5bbe42bb9d829e1d552b307d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:41.184093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25210f-5279-414f-9611-558910bfab45 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:41.194443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac25210f-5279-414f-9611-558910bfab45 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:41.215771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:41.256353 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fde80c-796d-46ae-9602-1d9b62115c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:41.256953 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde80c-796d-46ae-9602-1d9b62115c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:41.257227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde80c-796d-46ae-9602-1d9b62115c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:41.257493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fde80c-796d-46ae-9602-1d9b62115c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:41.258127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fde80c-796d-46ae-9602-1d9b62115c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:41.259752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fde80c-796d-46ae-9602-1d9b62115c08 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:41.270268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1079/2147] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:41.274679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545e5a7-b4e3-4be6-ad60-bf523566dd30 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:41.280369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:41.287416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d545e5a7-b4e3-4be6-ad60-bf523566dd30 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:41.292689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e69efa-bdc9-496f-8656-e5f67a83e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:41.293367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e69efa-bdc9-496f-8656-e5f67a83e4d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:41.293643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e69efa-bdc9-496f-8656-e5f67a83e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:41.293909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e69efa-bdc9-496f-8656-e5f67a83e4d2 admin admin] PATH_INFO: `/v3/projects/f90ae5d820bb4459b2934efbf3a007c9/users/25b5239a5bbe42bb9d829e1d552b307d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:41.295803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e69efa-bdc9-496f-8656-e5f67a83e4d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f90ae5d820bb4459b2934efbf3a007c9, user_id=25b5239a5bbe42bb9d829e1d552b307d, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:41.298984 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e69efa-bdc9-496f-8656-e5f67a83e4d2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:41.309447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1069/2148] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:41 2022] PUT /identity/v3/projects/f90ae5d820bb4459b2934efbf3a007c9/users/25b5239a5bbe42bb9d829e1d552b307d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:41.319909 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab890b-94f8-4db1-8830-ad90d61c67db None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:41.320258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab890b-94f8-4db1-8830-ad90d61c67db None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:41.320594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab890b-94f8-4db1-8830-ad90d61c67db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:41.348814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f2ab890b-94f8-4db1-8830-ad90d61c67db None None] MFA Rules not processed for user `25b5239a5bbe42bb9d829e1d552b307d`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:41.350004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f2ab890b-94f8-4db1-8830-ad90d61c67db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:41.418862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1080/2149] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:41.430807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e69efa-bdc9-496f-8656-e5f67a83e4d2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:41.432647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-73e69efa-bdc9-496f-8656-e5f67a83e4d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:41.491635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e69efa-bdc9-496f-8656-e5f67a83e4d2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:41.495202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6d9f8f-e0ec-479c-92b5-1bfcef5b6456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:41.495825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d9f8f-e0ec-479c-92b5-1bfcef5b6456 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:41.496094 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d9f8f-e0ec-479c-92b5-1bfcef5b6456 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:41.496358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6d9f8f-e0ec-479c-92b5-1bfcef5b6456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:41.505623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d9f8f-e0ec-479c-92b5-1bfcef5b6456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:41.507040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6d9f8f-e0ec-479c-92b5-1bfcef5b6456 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:41.517331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1070/2150] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:41 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:15:41.585306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab890b-94f8-4db1-8830-ad90d61c67db None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:41.667979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab890b-94f8-4db1-8830-ad90d61c67db None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:41.673019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:41.673863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:41.674249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:41.674718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] PATH_INFO: `/v3/users/5292f230d5924eaf91f5a6ab0887e78a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:41.676318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5292f230d5924eaf91f5a6ab0887e78a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:41.678343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:41.734379 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5292f230d5924eaf91f5a6ab0887e78a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:41.736931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:41.752582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5292f230d5924eaf91f5a6ab0887e78a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:41.753857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5292f230d5924eaf91f5a6ab0887e78a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:41.754472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5292f230d5924eaf91f5a6ab0887e78a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:41.763780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5292f230d5924eaf91f5a6ab0887e78a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:41.806503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1081/2151] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:41 2022] DELETE /identity/v3/users/5292f230d5924eaf91f5a6ab0887e78a => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:15:41.811114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7883dc3-62ec-417b-85e9-3b51eae6ded1 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:41.811394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7883dc3-62ec-417b-85e9-3b51eae6ded1 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:41.812796 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7883dc3-62ec-417b-85e9-3b51eae6ded1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:41.841134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a7883dc3-62ec-417b-85e9-3b51eae6ded1 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:41.842369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a7883dc3-62ec-417b-85e9-3b51eae6ded1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:41.918930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1071/2152] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:15:41.929306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:41.930747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:41.994163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f941c7-0ae7-464e-893c-eb5a774c9cc5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:41.998929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb23c4-92d0-478a-8fc5-cef614d67c78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:42.000061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb23c4-92d0-478a-8fc5-cef614d67c78 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:42.000508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb23c4-92d0-478a-8fc5-cef614d67c78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:42.000961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb23c4-92d0-478a-8fc5-cef614d67c78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:42.014525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb23c4-92d0-478a-8fc5-cef614d67c78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:42.016026 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb23c4-92d0-478a-8fc5-cef614d67c78 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:42.026297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1082/2153] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:42.366281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7883dc3-62ec-417b-85e9-3b51eae6ded1 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:42.369816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a7883dc3-62ec-417b-85e9-3b51eae6ded1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:42.386221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7883dc3-62ec-417b-85e9-3b51eae6ded1 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:42.391355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:42.394018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:42.394730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:42.394730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] PATH_INFO: `/v3/projects/6d59b0d6f1dd4974be3f7c0e6349bb57` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:42.395844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d59b0d6f1dd4974be3f7c0e6349bb57)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:42.397759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:42.464087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d59b0d6f1dd4974be3f7c0e6349bb57', 'request_id': 'req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:42.473540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d59b0d6f1dd4974be3f7c0e6349bb57', 'request_id': 'req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:42.479923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d59b0d6f1dd4974be3f7c0e6349bb57', 'request_id': 'req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:42.492817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] The token cache is being invalidate because project 6d59b0d6f1dd4974be3f7c0e6349bb57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:42.493110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:42.493762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:42.495250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1072/2154] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:42 2022] DELETE /identity/v3/projects/6d59b0d6f1dd4974be3f7c0e6349bb57 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:42.499243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb23c4-92d0-478a-8fc5-cef614d67c78 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:42.509358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb23c4-92d0-478a-8fc5-cef614d67c78 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:42.568770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:42.569494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:42.569844 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:42.570186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] PATH_INFO: `/v3/users/25b5239a5bbe42bb9d829e1d552b307d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:42.571371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=25b5239a5bbe42bb9d829e1d552b307d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:42.573405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:42.633459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25b5239a5bbe42bb9d829e1d552b307d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:42.640443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25b5239a5bbe42bb9d829e1d552b307d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:42.641075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25b5239a5bbe42bb9d829e1d552b307d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:42.642186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25b5239a5bbe42bb9d829e1d552b307d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:42.649985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-36ed43f6-f5b4-4fb2-b724-1e8d7a9ace0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25b5239a5bbe42bb9d829e1d552b307d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:42.656989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:42.671872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1083/2155] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:15:42 2022] DELETE /identity/v3/users/25b5239a5bbe42bb9d829e1d552b307d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:43.129854 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:43.131568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:43.155886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdd3d19-b6a9-4566-987a-10c3cb1ef893 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:43.170616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:43.230727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:43.231543 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.231826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.232848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] PATH_INFO: `/v3/projects/f90ae5d820bb4459b2934efbf3a007c9` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.234255 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=f90ae5d820bb4459b2934efbf3a007c9)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:43.236826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:43.284445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f90ae5d820bb4459b2934efbf3a007c9', 'request_id': 'req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:43.291336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f90ae5d820bb4459b2934efbf3a007c9', 'request_id': 'req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:43.296787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f90ae5d820bb4459b2934efbf3a007c9', 'request_id': 'req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:43.308318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] The token cache is being invalidate because project f90ae5d820bb4459b2934efbf3a007c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:15:43.308604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:43.309240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0987a30-5618-4ef9-8ad2-b1f08bd6f1ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:15:43.310707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1073/2156] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:15:43 2022] DELETE /identity/v3/projects/f90ae5d820bb4459b2934efbf3a007c9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:43.316330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e138a5-47f8-4816-a543-bff18162694d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.316611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e138a5-47f8-4816-a543-bff18162694d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.316883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e138a5-47f8-4816-a543-bff18162694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.338074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-03e138a5-47f8-4816-a543-bff18162694d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:43.339265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-03e138a5-47f8-4816-a543-bff18162694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:43.400265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1084/2157] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:43.408269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d37f51-cbed-4af6-bd2e-7e9f55d0ac0a None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.408644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d37f51-cbed-4af6-bd2e-7e9f55d0ac0a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.408958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d37f51-cbed-4af6-bd2e-7e9f55d0ac0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.434227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-97d37f51-cbed-4af6-bd2e-7e9f55d0ac0a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:43.435435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-97d37f51-cbed-4af6-bd2e-7e9f55d0ac0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:43.445665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1074/2158] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:43.451686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e138a5-47f8-4816-a543-bff18162694d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:43.462056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e138a5-47f8-4816-a543-bff18162694d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:43.467163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32e5a6-1e46-4216-8a77-87e65f9aa8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:43.467831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32e5a6-1e46-4216-8a77-87e65f9aa8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.468393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32e5a6-1e46-4216-8a77-87e65f9aa8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.468672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32e5a6-1e46-4216-8a77-87e65f9aa8b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.469325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32e5a6-1e46-4216-8a77-87e65f9aa8b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:15:43.469628 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32e5a6-1e46-4216-8a77-87e65f9aa8b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:43.471322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32e5a6-1e46-4216-8a77-87e65f9aa8b9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:43.484746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1085/2159] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:15:43.489345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d37f51-cbed-4af6-bd2e-7e9f55d0ac0a None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:43.498867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d37f51-cbed-4af6-bd2e-7e9f55d0ac0a None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:43.504314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece8b13-e1d8-42ac-8097-6300b6df9c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:43.504908 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece8b13-e1d8-42ac-8097-6300b6df9c53 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.505188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece8b13-e1d8-42ac-8097-6300b6df9c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.505460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece8b13-e1d8-42ac-8097-6300b6df9c53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.506099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece8b13-e1d8-42ac-8097-6300b6df9c53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:43.508060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece8b13-e1d8-42ac-8097-6300b6df9c53 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:43.539096 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1075/2160] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:15:43 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:15:43.543391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32e5a6-1e46-4216-8a77-87e65f9aa8b9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:43.555844 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32e5a6-1e46-4216-8a77-87e65f9aa8b9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:43.615827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe10c49-3c20-4110-8f6b-f27aed596000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:43.616431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe10c49-3c20-4110-8f6b-f27aed596000 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.616708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe10c49-3c20-4110-8f6b-f27aed596000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.617004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe10c49-3c20-4110-8f6b-f27aed596000 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.617630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe10c49-3c20-4110-8f6b-f27aed596000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:43.619622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe10c49-3c20-4110-8f6b-f27aed596000 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:43.641895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1086/2161] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:15:43 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:15:43.646029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece8b13-e1d8-42ac-8097-6300b6df9c53 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:43.656299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece8b13-e1d8-42ac-8097-6300b6df9c53 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:43.660903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e8aa9-69b7-44d8-8236-78ed750b26d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:43.661495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e8aa9-69b7-44d8-8236-78ed750b26d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.661777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e8aa9-69b7-44d8-8236-78ed750b26d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.662049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5e8aa9-69b7-44d8-8236-78ed750b26d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.662695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e8aa9-69b7-44d8-8236-78ed750b26d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:43.664331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5e8aa9-69b7-44d8-8236-78ed750b26d9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:43.674620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1076/2162] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:43.678476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe10c49-3c20-4110-8f6b-f27aed596000 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:43.689407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe10c49-3c20-4110-8f6b-f27aed596000 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:43.694257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55fbe5-e126-494d-a2c8-375c6bbfc080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:43.694841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55fbe5-e126-494d-a2c8-375c6bbfc080 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.695117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55fbe5-e126-494d-a2c8-375c6bbfc080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.695386 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55fbe5-e126-494d-a2c8-375c6bbfc080 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.696057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55fbe5-e126-494d-a2c8-375c6bbfc080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:43.697674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55fbe5-e126-494d-a2c8-375c6bbfc080 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:43.708121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1087/2163] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:15:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:15:43.712207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e8aa9-69b7-44d8-8236-78ed750b26d9 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:43.723463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5e8aa9-69b7-44d8-8236-78ed750b26d9 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:43.728252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3153fbc-6fd3-45d2-bd29-acd8bd37f0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:43.728906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3153fbc-6fd3-45d2-bd29-acd8bd37f0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.729181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3153fbc-6fd3-45d2-bd29-acd8bd37f0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.729455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3153fbc-6fd3-45d2-bd29-acd8bd37f0f3 admin admin] PATH_INFO: `/v3/projects/b0c1ce10713049ee84be637c5ee9fc82/users/5ed224553bf342199843fe66f5e094da/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.744882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3153fbc-6fd3-45d2-bd29-acd8bd37f0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0c1ce10713049ee84be637c5ee9fc82, user_id=5ed224553bf342199843fe66f5e094da, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:43.747565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3153fbc-6fd3-45d2-bd29-acd8bd37f0f3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:43.761000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1077/2164] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:15:43 2022] PUT /identity/v3/projects/b0c1ce10713049ee84be637c5ee9fc82/users/5ed224553bf342199843fe66f5e094da/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:15:43.776762 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66ed35-adcd-4bc9-b8e5-f5bebce1b8bb None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.777052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66ed35-adcd-4bc9-b8e5-f5bebce1b8bb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.777355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b66ed35-adcd-4bc9-b8e5-f5bebce1b8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.807604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0b66ed35-adcd-4bc9-b8e5-f5bebce1b8bb None None] MFA Rules not processed for user `5ed224553bf342199843fe66f5e094da`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:43.808816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0b66ed35-adcd-4bc9-b8e5-f5bebce1b8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:43.849088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:43.876498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1088/2165] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:43.887670 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3153fbc-6fd3-45d2-bd29-acd8bd37f0f3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:15:43.889351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a3153fbc-6fd3-45d2-bd29-acd8bd37f0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:43.936170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:43.971679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3153fbc-6fd3-45d2-bd29-acd8bd37f0f3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:15:43.975801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e5ca9-4d23-4c22-8348-7c9f2f34a323 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf75b583add44b6b87bff5dc67080d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:15:43.976406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e5ca9-4d23-4c22-8348-7c9f2f34a323 service glance] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:43.976684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e5ca9-4d23-4c22-8348-7c9f2f34a323 service glance] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:43.976958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e5ca9-4d23-4c22-8348-7c9f2f34a323 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:43.986244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e5ca9-4d23-4c22-8348-7c9f2f34a323 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:15:43.987909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9e5ca9-4d23-4c22-8348-7c9f2f34a323 service glance] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:15:43.996910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1078/2166] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:15:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:15:48.587287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d0895-f76e-408e-b029-defb3fc23369 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:15:48.587886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d0895-f76e-408e-b029-defb3fc23369 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:15:48.587886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d0895-f76e-408e-b029-defb3fc23369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:15:48.607109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3b0d0895-f76e-408e-b029-defb3fc23369 None None] MFA Rules not processed for user `bae66efbdf994be2b6f49ba3e5c08ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:15:48.608539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3b0d0895-f76e-408e-b029-defb3fc23369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:15:48.666044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1089/2167] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:15:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:15:53.992893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=16 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:15:58.659267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=33 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:24.053771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e5ca9-4d23-4c22-8348-7c9f2f34a323 service glance] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:24.116005 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9e5ca9-4d23-4c22-8348-7c9f2f34a323 service glance] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:24.121017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:24.121631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:24.121905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:24.122175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] PATH_INFO: `/v3/users/5ed224553bf342199843fe66f5e094da` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:24.123284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ed224553bf342199843fe66f5e094da)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:24.125490 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:24.171826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ed224553bf342199843fe66f5e094da'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:24.172495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ed224553bf342199843fe66f5e094da'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:24.180261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ed224553bf342199843fe66f5e094da'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:24.187250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ed224553bf342199843fe66f5e094da'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:24.187925 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ed224553bf342199843fe66f5e094da'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:24.206413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1079/2168] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:16:24 2022] DELETE /identity/v3/users/5ed224553bf342199843fe66f5e094da => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:16:24.211202 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ed6a-4d09-4cde-804d-fe6a0cfaf8e5 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:24.211807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ed6a-4d09-4cde-804d-fe6a0cfaf8e5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:24.211960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7ed6a-4d09-4cde-804d-fe6a0cfaf8e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:24.232348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8cc7ed6a-4d09-4cde-804d-fe6a0cfaf8e5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:24.233811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8cc7ed6a-4d09-4cde-804d-fe6a0cfaf8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:24.293378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1090/2169] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:24.303810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:24.305197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:24.363040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a9a00-bb95-4924-b624-8202d3860b2a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:24.367583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68635254-635a-484c-856d-377ba9eb374a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:24.368181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68635254-635a-484c-856d-377ba9eb374a service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:24.368460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68635254-635a-484c-856d-377ba9eb374a service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:24.368727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68635254-635a-484c-856d-377ba9eb374a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:24.377601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68635254-635a-484c-856d-377ba9eb374a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:24.379037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68635254-635a-484c-856d-377ba9eb374a service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:24.388905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1080/2170] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:16:24.678078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7ed6a-4d09-4cde-804d-fe6a0cfaf8e5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:24.679521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8cc7ed6a-4d09-4cde-804d-fe6a0cfaf8e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:24.691501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7ed6a-4d09-4cde-804d-fe6a0cfaf8e5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:24.696747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:24.697375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:24.697651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:24.697921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] PATH_INFO: `/v3/projects/b0c1ce10713049ee84be637c5ee9fc82` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:24.699037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0c1ce10713049ee84be637c5ee9fc82)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:24.701286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:24.750067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0c1ce10713049ee84be637c5ee9fc82', 'request_id': 'req-9c0eb004-30dc-4495-8a39-18933814e3a1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:24.756279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0c1ce10713049ee84be637c5ee9fc82', 'request_id': 'req-9c0eb004-30dc-4495-8a39-18933814e3a1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:24.763273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0c1ce10713049ee84be637c5ee9fc82', 'request_id': 'req-9c0eb004-30dc-4495-8a39-18933814e3a1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:24.774246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] The token cache is being invalidate because project b0c1ce10713049ee84be637c5ee9fc82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:16:24.774532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:24.775169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c0eb004-30dc-4495-8a39-18933814e3a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:24.776661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1091/2171] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:16:24 2022] DELETE /identity/v3/projects/b0c1ce10713049ee84be637c5ee9fc82 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:16:24.781578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1865e3-31f2-406c-bfc4-4f45aaf1815d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:24.781862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1865e3-31f2-406c-bfc4-4f45aaf1815d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:24.782166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1865e3-31f2-406c-bfc4-4f45aaf1815d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:24.802072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-ae1865e3-31f2-406c-bfc4-4f45aaf1815d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:24.803180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ae1865e3-31f2-406c-bfc4-4f45aaf1815d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:24.820556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:24.866368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1081/2172] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:24.874053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5faf16-dd1c-4101-aac6-bf93a53b0844 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:24.874339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5faf16-dd1c-4101-aac6-bf93a53b0844 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:24.874613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5faf16-dd1c-4101-aac6-bf93a53b0844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:24.895289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-bd5faf16-dd1c-4101-aac6-bf93a53b0844 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:24.896499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bd5faf16-dd1c-4101-aac6-bf93a53b0844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:24.904441 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1092/2173] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:24.908859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1865e3-31f2-406c-bfc4-4f45aaf1815d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:24.919020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1865e3-31f2-406c-bfc4-4f45aaf1815d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:24.923740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536acfb-f9f7-467e-9041-5fa52a73a99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:24.924340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536acfb-f9f7-467e-9041-5fa52a73a99c admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:24.924611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536acfb-f9f7-467e-9041-5fa52a73a99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:24.924876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b536acfb-f9f7-467e-9041-5fa52a73a99c admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:24.925513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536acfb-f9f7-467e-9041-5fa52a73a99c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:16:24.926322 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536acfb-f9f7-467e-9041-5fa52a73a99c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:24.927937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b536acfb-f9f7-467e-9041-5fa52a73a99c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:24.939478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1082/2174] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:16:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:16:24.944745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5faf16-dd1c-4101-aac6-bf93a53b0844 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:24.954025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5faf16-dd1c-4101-aac6-bf93a53b0844 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:24.958788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09415c-cac4-4b6f-a2b1-5938b064e841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:24.959382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09415c-cac4-4b6f-a2b1-5938b064e841 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:24.959735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09415c-cac4-4b6f-a2b1-5938b064e841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:24.960006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be09415c-cac4-4b6f-a2b1-5938b064e841 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:24.960642 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09415c-cac4-4b6f-a2b1-5938b064e841 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:24.962566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be09415c-cac4-4b6f-a2b1-5938b064e841 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:24.997022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1093/2175] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:16:24 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:16:24.998449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536acfb-f9f7-467e-9041-5fa52a73a99c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:25.008738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b536acfb-f9f7-467e-9041-5fa52a73a99c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:25.066312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbb33f-69a6-4ec8-a6bf-c44e15125d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:25.066983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbb33f-69a6-4ec8-a6bf-c44e15125d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:25.067265 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbb33f-69a6-4ec8-a6bf-c44e15125d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:25.067573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fbb33f-69a6-4ec8-a6bf-c44e15125d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:25.068206 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbb33f-69a6-4ec8-a6bf-c44e15125d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:25.070110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fbb33f-69a6-4ec8-a6bf-c44e15125d9b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:25.090452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1083/2176] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:16:24 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:16:25.094604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09415c-cac4-4b6f-a2b1-5938b064e841 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:25.105549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be09415c-cac4-4b6f-a2b1-5938b064e841 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:25.110577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f75ec-af19-43b6-8184-5830fd41d1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:25.111213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f75ec-af19-43b6-8184-5830fd41d1ca admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:25.111534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f75ec-af19-43b6-8184-5830fd41d1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:25.111809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f75ec-af19-43b6-8184-5830fd41d1ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:25.112456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f75ec-af19-43b6-8184-5830fd41d1ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:25.114681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0f75ec-af19-43b6-8184-5830fd41d1ca admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:25.125285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1094/2177] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:16:25.129334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbb33f-69a6-4ec8-a6bf-c44e15125d9b admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:25.139247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fbb33f-69a6-4ec8-a6bf-c44e15125d9b admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:25.143903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b9240-545f-4013-8a53-62f1454d19ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:25.144498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b9240-545f-4013-8a53-62f1454d19ff admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:25.144803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b9240-545f-4013-8a53-62f1454d19ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:25.145075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b9240-545f-4013-8a53-62f1454d19ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:25.145713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b9240-545f-4013-8a53-62f1454d19ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:25.147339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562b9240-545f-4013-8a53-62f1454d19ff admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:25.157775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1084/2178] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:16:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:16:25.162317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f75ec-af19-43b6-8184-5830fd41d1ca admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:25.172131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f75ec-af19-43b6-8184-5830fd41d1ca admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:25.177569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f786573-985c-40c4-9fb9-d15bd8665fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:25.178196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f786573-985c-40c4-9fb9-d15bd8665fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:25.178475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f786573-985c-40c4-9fb9-d15bd8665fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:25.178751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f786573-985c-40c4-9fb9-d15bd8665fcc admin admin] PATH_INFO: `/v3/projects/5e8b6b3f599646be926ca705a7484879/users/2383576decde451a882dae53a0a9e3ae/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:25.192989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f786573-985c-40c4-9fb9-d15bd8665fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8b6b3f599646be926ca705a7484879, user_id=2383576decde451a882dae53a0a9e3ae, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:25.195695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f786573-985c-40c4-9fb9-d15bd8665fcc admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:25.208078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1095/2179] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:16:25 2022] PUT /identity/v3/projects/5e8b6b3f599646be926ca705a7484879/users/2383576decde451a882dae53a0a9e3ae/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:16:25.219421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cca37a-48a0-4c2c-8c19-619cee4ef504 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:25.219741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cca37a-48a0-4c2c-8c19-619cee4ef504 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:25.220016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cca37a-48a0-4c2c-8c19-619cee4ef504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:25.249371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-80cca37a-48a0-4c2c-8c19-619cee4ef504 None None] MFA Rules not processed for user `2383576decde451a882dae53a0a9e3ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:25.250587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-80cca37a-48a0-4c2c-8c19-619cee4ef504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:25.327414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1085/2180] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:16:25.362129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f786573-985c-40c4-9fb9-d15bd8665fcc admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:25.364028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6f786573-985c-40c4-9fb9-d15bd8665fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:25.393232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:25.431774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f786573-985c-40c4-9fb9-d15bd8665fcc admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:25.437433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724bd820-f888-43b4-883e-23ff4dd5a779 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:25.438053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bd820-f888-43b4-883e-23ff4dd5a779 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:25.438341 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bd820-f888-43b4-883e-23ff4dd5a779 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:25.438601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724bd820-f888-43b4-883e-23ff4dd5a779 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:25.448343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bd820-f888-43b4-883e-23ff4dd5a779 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:25.449784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724bd820-f888-43b4-883e-23ff4dd5a779 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:25.459410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1096/2181] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:16:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:16:25.475510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cca37a-48a0-4c2c-8c19-619cee4ef504 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:25.495155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:25.540825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cca37a-48a0-4c2c-8c19-619cee4ef504 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:25.545395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:25.546027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:25.546308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:25.546580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] PATH_INFO: `/v3/users/2383576decde451a882dae53a0a9e3ae` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:25.547748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2383576decde451a882dae53a0a9e3ae)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:25.549717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:25.599519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2383576decde451a882dae53a0a9e3ae'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:25.600180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2383576decde451a882dae53a0a9e3ae'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:25.608941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2383576decde451a882dae53a0a9e3ae'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:25.616561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2383576decde451a882dae53a0a9e3ae'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:25.617265 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2383576decde451a882dae53a0a9e3ae'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:25.638337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1086/2182] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:16:25 2022] DELETE /identity/v3/users/2383576decde451a882dae53a0a9e3ae => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:16:25.643079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832f3b4-1f0e-4207-a5a2-7f1e374ec418 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:25.643375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832f3b4-1f0e-4207-a5a2-7f1e374ec418 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:25.643683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832f3b4-1f0e-4207-a5a2-7f1e374ec418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:25.665510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0832f3b4-1f0e-4207-a5a2-7f1e374ec418 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:25.666720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0832f3b4-1f0e-4207-a5a2-7f1e374ec418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:25.743094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1097/2183] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:25.753227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:25.755239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:25.812787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f7bb05-e703-47e5-bbe1-ff157f10188b admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:25.816585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8018b349-192c-4fcb-81b4-01c803a968e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:25.817280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018b349-192c-4fcb-81b4-01c803a968e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:25.817553 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018b349-192c-4fcb-81b4-01c803a968e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:25.817822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8018b349-192c-4fcb-81b4-01c803a968e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:25.827789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018b349-192c-4fcb-81b4-01c803a968e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:25.829336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8018b349-192c-4fcb-81b4-01c803a968e3 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:25.838833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1087/2184] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:16:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:16:26.184071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832f3b4-1f0e-4207-a5a2-7f1e374ec418 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:26.185904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0832f3b4-1f0e-4207-a5a2-7f1e374ec418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:26.199113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832f3b4-1f0e-4207-a5a2-7f1e374ec418 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:26.204085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:26.204744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:26.204992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:26.205266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] PATH_INFO: `/v3/projects/5e8b6b3f599646be926ca705a7484879` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:26.212715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e8b6b3f599646be926ca705a7484879)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:26.216163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:26.281563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e8b6b3f599646be926ca705a7484879', 'request_id': 'req-434163b2-b4d4-4f46-be23-f6f83a2794a8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:26.283950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e8b6b3f599646be926ca705a7484879', 'request_id': 'req-434163b2-b4d4-4f46-be23-f6f83a2794a8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:26.293075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e8b6b3f599646be926ca705a7484879', 'request_id': 'req-434163b2-b4d4-4f46-be23-f6f83a2794a8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:26.309867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] The token cache is being invalidate because project 5e8b6b3f599646be926ca705a7484879 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:16:26.310159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:26.310878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-434163b2-b4d4-4f46-be23-f6f83a2794a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:26.312398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1098/2185] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:16:26 2022] DELETE /identity/v3/projects/5e8b6b3f599646be926ca705a7484879 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:16:26.318831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57857a51-f97b-4520-bd70-02e19c8b8ac1 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:26.318831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57857a51-f97b-4520-bd70-02e19c8b8ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:26.318831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57857a51-f97b-4520-bd70-02e19c8b8ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:26.364237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-57857a51-f97b-4520-bd70-02e19c8b8ac1 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:26.366791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-57857a51-f97b-4520-bd70-02e19c8b8ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:26.417969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:26.471847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1088/2186] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:26.475466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d3cc-a9cd-4933-a451-a2e7907082b9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:26.475948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d3cc-a9cd-4933-a451-a2e7907082b9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:26.476065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a68d3cc-a9cd-4933-a451-a2e7907082b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:26.499367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7a68d3cc-a9cd-4933-a451-a2e7907082b9 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:26.500655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7a68d3cc-a9cd-4933-a451-a2e7907082b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:26.513539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1099/2187] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:26.520580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57857a51-f97b-4520-bd70-02e19c8b8ac1 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:26.530379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57857a51-f97b-4520-bd70-02e19c8b8ac1 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:26.535934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431635e9-d4d9-473c-8476-5ebeb1e41f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:26.536835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431635e9-d4d9-473c-8476-5ebeb1e41f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:26.537116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431635e9-d4d9-473c-8476-5ebeb1e41f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:26.537382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431635e9-d4d9-473c-8476-5ebeb1e41f89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:26.538033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431635e9-d4d9-473c-8476-5ebeb1e41f89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:16:26.538330 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431635e9-d4d9-473c-8476-5ebeb1e41f89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:26.540450 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431635e9-d4d9-473c-8476-5ebeb1e41f89 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:26.553041 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1089/2188] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:16:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:16:26.557304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d3cc-a9cd-4933-a451-a2e7907082b9 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:26.568011 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a68d3cc-a9cd-4933-a451-a2e7907082b9 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:26.573034 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508f91d-a9a8-4dc0-86c5-7b842f77a3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:26.573715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f91d-a9a8-4dc0-86c5-7b842f77a3f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:26.574196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f91d-a9a8-4dc0-86c5-7b842f77a3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:26.574450 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4508f91d-a9a8-4dc0-86c5-7b842f77a3f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:26.575229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508f91d-a9a8-4dc0-86c5-7b842f77a3f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:26.578012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4508f91d-a9a8-4dc0-86c5-7b842f77a3f5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:26.612868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1100/2189] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:16:26 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:16:26.618057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431635e9-d4d9-473c-8476-5ebeb1e41f89 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:26.628159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431635e9-d4d9-473c-8476-5ebeb1e41f89 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:26.711790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d20e-c4f2-4982-bb05-5465b79ae0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:26.712419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d20e-c4f2-4982-bb05-5465b79ae0ba admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:26.712684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d20e-c4f2-4982-bb05-5465b79ae0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:26.712956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3d20e-c4f2-4982-bb05-5465b79ae0ba admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:26.713591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3d20e-c4f2-4982-bb05-5465b79ae0ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:26.715583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a3d20e-c4f2-4982-bb05-5465b79ae0ba admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:26.750754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1090/2190] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:16:26 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:16:26.752218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508f91d-a9a8-4dc0-86c5-7b842f77a3f5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:26.764334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4508f91d-a9a8-4dc0-86c5-7b842f77a3f5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:26.772957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb58e1e-1211-46d3-be0b-9da7e2fb40f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:26.773645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb58e1e-1211-46d3-be0b-9da7e2fb40f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:26.774002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb58e1e-1211-46d3-be0b-9da7e2fb40f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:26.774474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb58e1e-1211-46d3-be0b-9da7e2fb40f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:26.775142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb58e1e-1211-46d3-be0b-9da7e2fb40f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:26.776792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb58e1e-1211-46d3-be0b-9da7e2fb40f7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:26.790703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1101/2191] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:16:26.791900 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d20e-c4f2-4982-bb05-5465b79ae0ba admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:26.802772 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a3d20e-c4f2-4982-bb05-5465b79ae0ba admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:26.807598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7d3ee-2c15-4c18-bd36-09d114e9510c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:26.808198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7d3ee-2c15-4c18-bd36-09d114e9510c admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:26.808509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7d3ee-2c15-4c18-bd36-09d114e9510c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:26.808780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7d3ee-2c15-4c18-bd36-09d114e9510c admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:26.809423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7d3ee-2c15-4c18-bd36-09d114e9510c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:26.811019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7d3ee-2c15-4c18-bd36-09d114e9510c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:26.824655 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1091/2192] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:16:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:16:26.828750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb58e1e-1211-46d3-be0b-9da7e2fb40f7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:26.839466 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb58e1e-1211-46d3-be0b-9da7e2fb40f7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:26.844287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7bd88-b4a5-409e-a624-926b4d470a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:26.844937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7bd88-b4a5-409e-a624-926b4d470a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:26.845632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7bd88-b4a5-409e-a624-926b4d470a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:26.845632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7bd88-b4a5-409e-a624-926b4d470a4c admin admin] PATH_INFO: `/v3/projects/d8dde61b4f3943738e0f56aec9ad577a/users/808a101a9e8447d9a01e251e7287b330/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:26.848361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:26.864387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7bd88-b4a5-409e-a624-926b4d470a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8dde61b4f3943738e0f56aec9ad577a, user_id=808a101a9e8447d9a01e251e7287b330, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:26.867562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7bd88-b4a5-409e-a624-926b4d470a4c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:26.879862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1102/2193] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:16:26 2022] PUT /identity/v3/projects/d8dde61b4f3943738e0f56aec9ad577a/users/808a101a9e8447d9a01e251e7287b330/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:16:26.890038 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b43ca-a8db-480b-8173-1d4155d029e0 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:26.890351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b43ca-a8db-480b-8173-1d4155d029e0 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:26.890628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b43ca-a8db-480b-8173-1d4155d029e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:26.918971 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-810b43ca-a8db-480b-8173-1d4155d029e0 None None] MFA Rules not processed for user `808a101a9e8447d9a01e251e7287b330`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:26.920225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-810b43ca-a8db-480b-8173-1d4155d029e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:26.984305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1092/2194] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:26.995482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7bd88-b4a5-409e-a624-926b4d470a4c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:26.997200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-34a7bd88-b4a5-409e-a624-926b4d470a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:27.053221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7bd88-b4a5-409e-a624-926b4d470a4c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:27.056880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5ee70-1dfc-4ec7-a5f6-6489cde00ffc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:27.057474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ee70-1dfc-4ec7-a5f6-6489cde00ffc service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:27.057751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ee70-1dfc-4ec7-a5f6-6489cde00ffc service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:27.058022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5ee70-1dfc-4ec7-a5f6-6489cde00ffc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:27.067324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5ee70-1dfc-4ec7-a5f6-6489cde00ffc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:27.068793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5ee70-1dfc-4ec7-a5f6-6489cde00ffc service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:27.077537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1103/2195] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:16:30.109317 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b43ca-a8db-480b-8173-1d4155d029e0 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:30.157012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:30.182630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b43ca-a8db-480b-8173-1d4155d029e0 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:30.187257 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:30.187917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:30.188193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:30.188459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] PATH_INFO: `/v3/users/808a101a9e8447d9a01e251e7287b330` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:30.189766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] RBAC: Authorizing `identity:delete_user(user_id=808a101a9e8447d9a01e251e7287b330)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:30.191880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:30.250065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '808a101a9e8447d9a01e251e7287b330'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:30.250866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '808a101a9e8447d9a01e251e7287b330'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:30.260788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '808a101a9e8447d9a01e251e7287b330'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:30.269563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '808a101a9e8447d9a01e251e7287b330'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:30.271638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '808a101a9e8447d9a01e251e7287b330'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:30.290364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1093/2196] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:16:30 2022] DELETE /identity/v3/users/808a101a9e8447d9a01e251e7287b330 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:16:30.294637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d1564-0cfc-4682-b4d9-8332c3769a18 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:30.295123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d1564-0cfc-4682-b4d9-8332c3769a18 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:30.295228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d1564-0cfc-4682-b4d9-8332c3769a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:30.321014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4d2d1564-0cfc-4682-b4d9-8332c3769a18 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:30.322690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4d2d1564-0cfc-4682-b4d9-8332c3769a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:30.393955 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1104/2197] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:30.413297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:30.414884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:30.483618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e51b6d-c72f-4e9f-850f-53f610b54019 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:30.487973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a57eb-162a-4418-869f-f2b4405bd6fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:30.489163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a57eb-162a-4418-869f-f2b4405bd6fc service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:30.489519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a57eb-162a-4418-869f-f2b4405bd6fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:30.490466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a57eb-162a-4418-869f-f2b4405bd6fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:30.499849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a57eb-162a-4418-869f-f2b4405bd6fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:30.501385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a57eb-162a-4418-869f-f2b4405bd6fc service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:30.511919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1094/2198] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:16:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:16:31.073248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d1564-0cfc-4682-b4d9-8332c3769a18 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:31.074680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4d2d1564-0cfc-4682-b4d9-8332c3769a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:31.088529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d1564-0cfc-4682-b4d9-8332c3769a18 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:31.096279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:31.096279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.096504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.096723 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] PATH_INFO: `/v3/projects/d8dde61b4f3943738e0f56aec9ad577a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.098468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8dde61b4f3943738e0f56aec9ad577a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:31.100838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:31.169849 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8dde61b4f3943738e0f56aec9ad577a', 'request_id': 'req-74757e92-20cf-4221-a2f4-69254f84f171'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:31.176001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8dde61b4f3943738e0f56aec9ad577a', 'request_id': 'req-74757e92-20cf-4221-a2f4-69254f84f171'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:31.183594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8dde61b4f3943738e0f56aec9ad577a', 'request_id': 'req-74757e92-20cf-4221-a2f4-69254f84f171'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:31.196298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] The token cache is being invalidate because project d8dde61b4f3943738e0f56aec9ad577a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:16:31.196590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:31.197237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-74757e92-20cf-4221-a2f4-69254f84f171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:16:31.198969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1105/2199] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:16:31 2022] DELETE /identity/v3/projects/d8dde61b4f3943738e0f56aec9ad577a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:16:31.203631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64761306-0f12-469e-8097-56aa5f93c6f4 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.204068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64761306-0f12-469e-8097-56aa5f93c6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.204181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64761306-0f12-469e-8097-56aa5f93c6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.239610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-64761306-0f12-469e-8097-56aa5f93c6f4 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:31.241823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-64761306-0f12-469e-8097-56aa5f93c6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:31.297588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:31.311591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1095/2200] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:31.318888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef053ec6-31c3-4dc4-a3c8-0e1f8c4afeaf None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.319428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef053ec6-31c3-4dc4-a3c8-0e1f8c4afeaf None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.319428 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef053ec6-31c3-4dc4-a3c8-0e1f8c4afeaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.340577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ef053ec6-31c3-4dc4-a3c8-0e1f8c4afeaf None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:31.341765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ef053ec6-31c3-4dc4-a3c8-0e1f8c4afeaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:31.350011 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1106/2201] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:31.354404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64761306-0f12-469e-8097-56aa5f93c6f4 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:31.365026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64761306-0f12-469e-8097-56aa5f93c6f4 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:31.369695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b47384-083f-45dd-9390-80b17c97bdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:31.370304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b47384-083f-45dd-9390-80b17c97bdf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.370581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b47384-083f-45dd-9390-80b17c97bdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.370849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b47384-083f-45dd-9390-80b17c97bdf3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.371527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b47384-083f-45dd-9390-80b17c97bdf3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:16:31.371831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b47384-083f-45dd-9390-80b17c97bdf3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:31.373451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b47384-083f-45dd-9390-80b17c97bdf3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:31.385503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1096/2202] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:16:31.389694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef053ec6-31c3-4dc4-a3c8-0e1f8c4afeaf None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:31.395087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:31.404692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef053ec6-31c3-4dc4-a3c8-0e1f8c4afeaf None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:31.409822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f65b01-4567-449e-b64c-553dbe17864e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:31.411620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65b01-4567-449e-b64c-553dbe17864e admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.411962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65b01-4567-449e-b64c-553dbe17864e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.412260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f65b01-4567-449e-b64c-553dbe17864e admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.412918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f65b01-4567-449e-b64c-553dbe17864e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:31.414850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f65b01-4567-449e-b64c-553dbe17864e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:31.444754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1107/2203] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:16:31 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:16:31.448977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b47384-083f-45dd-9390-80b17c97bdf3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:31.461759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b47384-083f-45dd-9390-80b17c97bdf3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:31.533583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f53011-bb9d-4f62-a06b-f38616ad36b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:31.534187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53011-bb9d-4f62-a06b-f38616ad36b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.534469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53011-bb9d-4f62-a06b-f38616ad36b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.534743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f53011-bb9d-4f62-a06b-f38616ad36b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.536158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f53011-bb9d-4f62-a06b-f38616ad36b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:31.538116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f53011-bb9d-4f62-a06b-f38616ad36b4 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:31.561434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1097/2204] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:16:31 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:16:31.565448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f65b01-4567-449e-b64c-553dbe17864e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:31.576018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f65b01-4567-449e-b64c-553dbe17864e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:31.580578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0784b26-0a7a-4e56-bae0-42fc2b15be99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:31.581162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0784b26-0a7a-4e56-bae0-42fc2b15be99 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.581432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0784b26-0a7a-4e56-bae0-42fc2b15be99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.581699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0784b26-0a7a-4e56-bae0-42fc2b15be99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.582955 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0784b26-0a7a-4e56-bae0-42fc2b15be99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:31.584599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0784b26-0a7a-4e56-bae0-42fc2b15be99 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:31.594190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1108/2205] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:16:31.598583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f53011-bb9d-4f62-a06b-f38616ad36b4 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:31.609646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f53011-bb9d-4f62-a06b-f38616ad36b4 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:31.614411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b99fea-384e-44ef-a2ba-d097527fabef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:31.615012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b99fea-384e-44ef-a2ba-d097527fabef admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.615287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b99fea-384e-44ef-a2ba-d097527fabef admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.615598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b99fea-384e-44ef-a2ba-d097527fabef admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.616236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b99fea-384e-44ef-a2ba-d097527fabef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:31.617833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b99fea-384e-44ef-a2ba-d097527fabef admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:31.628295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1098/2206] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:16:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:16:31.632762 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0784b26-0a7a-4e56-bae0-42fc2b15be99 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:31.642759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0784b26-0a7a-4e56-bae0-42fc2b15be99 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:31.647966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48080976-ae1f-4d26-82c7-969ae6a98dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:31.648601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48080976-ae1f-4d26-82c7-969ae6a98dcb admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.648881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48080976-ae1f-4d26-82c7-969ae6a98dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.649155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48080976-ae1f-4d26-82c7-969ae6a98dcb admin admin] PATH_INFO: `/v3/projects/37278fff738c474d9090d3324f984711/users/ca1929f4d511416a93c784d327691573/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.664264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48080976-ae1f-4d26-82c7-969ae6a98dcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=37278fff738c474d9090d3324f984711, user_id=ca1929f4d511416a93c784d327691573, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:31.667006 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48080976-ae1f-4d26-82c7-969ae6a98dcb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:31.680743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1109/2207] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:16:31 2022] PUT /identity/v3/projects/37278fff738c474d9090d3324f984711/users/ca1929f4d511416a93c784d327691573/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:16:31.690821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992210af-6f54-443f-a5b1-f7bc5b064a50 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.691103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992210af-6f54-443f-a5b1-f7bc5b064a50 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.691679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992210af-6f54-443f-a5b1-f7bc5b064a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.719525 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-992210af-6f54-443f-a5b1-f7bc5b064a50 None None] MFA Rules not processed for user `ca1929f4d511416a93c784d327691573`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:31.720788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-992210af-6f54-443f-a5b1-f7bc5b064a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:31.784653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1099/2208] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:16:31.795467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48080976-ae1f-4d26-82c7-969ae6a98dcb admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:31.797145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-48080976-ae1f-4d26-82c7-969ae6a98dcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:31.852908 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48080976-ae1f-4d26-82c7-969ae6a98dcb admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:31.856518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7dd8d-d56b-4954-9f8c-c573079b8d21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:31.857112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7dd8d-d56b-4954-9f8c-c573079b8d21 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:31.857393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7dd8d-d56b-4954-9f8c-c573079b8d21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:31.857661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7dd8d-d56b-4954-9f8c-c573079b8d21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:31.867364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7dd8d-d56b-4954-9f8c-c573079b8d21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:31.868842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7dd8d-d56b-4954-9f8c-c573079b8d21 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:31.878817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1110/2209] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:16:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:16:33.230711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48802-99f9-481d-8028-677747b154c3 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:33.231216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48802-99f9-481d-8028-677747b154c3 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:33.231344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48802-99f9-481d-8028-677747b154c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:33.253309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-1cb48802-99f9-481d-8028-677747b154c3 None None] MFA Rules not processed for user `ca1929f4d511416a93c784d327691573`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:33.255283 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1cb48802-99f9-481d-8028-677747b154c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:33.262374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1100/2210] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:36.698297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db7e5-1d2b-4b48-864c-499dfa063b18 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:36.698824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db7e5-1d2b-4b48-864c-499dfa063b18 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:36.698824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7db7e5-1d2b-4b48-864c-499dfa063b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:36.717802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4f7db7e5-1d2b-4b48-864c-499dfa063b18 None None] MFA Rules not processed for user `8f96c9a2a49047b5b8abd3b2c1475a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:36.718987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4f7db7e5-1d2b-4b48-864c-499dfa063b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:36.777874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1111/2211] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:36 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:43.246164 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:44.584541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb48802-99f9-481d-8028-677747b154c3 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:44.586083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1cb48802-99f9-481d-8028-677747b154c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:44.663105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb48802-99f9-481d-8028-677747b154c3 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:44.669532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef430267-c791-482b-a069-f1c401ad1fc0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:44.670916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef430267-c791-482b-a069-f1c401ad1fc0 service placement] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:44.670916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef430267-c791-482b-a069-f1c401ad1fc0 service placement] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:44.670916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef430267-c791-482b-a069-f1c401ad1fc0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:44.672484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ef430267-c791-482b-a069-f1c401ad1fc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:44.685130 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef430267-c791-482b-a069-f1c401ad1fc0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:44.686785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef430267-c791-482b-a069-f1c401ad1fc0 service placement] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:44.698898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1101/2212] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:16:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:16:46.519978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07254e9-b143-468a-850c-4b63dc0a6d9e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:46.519978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07254e9-b143-468a-850c-4b63dc0a6d9e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:46.520864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07254e9-b143-468a-850c-4b63dc0a6d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:46.540307 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f07254e9-b143-468a-850c-4b63dc0a6d9e None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:46.541521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f07254e9-b143-468a-850c-4b63dc0a6d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:46.589559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:16:46.600344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1112/2213] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:16:46 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:46.643306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef430267-c791-482b-a069-f1c401ad1fc0 service placement] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:16:46.644866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ef430267-c791-482b-a069-f1c401ad1fc0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:46.713080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef430267-c791-482b-a069-f1c401ad1fc0 service placement] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:16:46.717963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da345e2c-5847-41f5-9090-a006a3cc9c6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:16:46.718560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da345e2c-5847-41f5-9090-a006a3cc9c6b service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:46.718837 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da345e2c-5847-41f5-9090-a006a3cc9c6b service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:46.719105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da345e2c-5847-41f5-9090-a006a3cc9c6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:46.727848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da345e2c-5847-41f5-9090-a006a3cc9c6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:16:46.729430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da345e2c-5847-41f5-9090-a006a3cc9c6b service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:16:46.739265 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1102/2214] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:16:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:16:53.327167 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ea16-06af-4497-85b7-a42c9736c4e5 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:16:53.327834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ea16-06af-4497-85b7-a42c9736c4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:16:53.327932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855ea16-06af-4497-85b7-a42c9736c4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:16:53.350272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5855ea16-06af-4497-85b7-a42c9736c4e5 None None] MFA Rules not processed for user `8f468d0b1fec4502858b8e29ac640ebd`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:16:53.351769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5855ea16-06af-4497-85b7-a42c9736c4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:16:53.432510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1113/2215] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:16:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:16:56.734967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=64 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:17:03.417381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=26 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:17:08.987910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5def-9930-4846-b9bc-b863bc25384d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:08.988465 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5def-9930-4846-b9bc-b863bc25384d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:08.988465 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae5def-9930-4846-b9bc-b863bc25384d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:09.008416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-06ae5def-9930-4846-b9bc-b863bc25384d None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:09.009629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-06ae5def-9930-4846-b9bc-b863bc25384d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:09.017508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1103/2216] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 10 14:17:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:09.032524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855ea16-06af-4497-85b7-a42c9736c4e5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:09.045163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855ea16-06af-4497-85b7-a42c9736c4e5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:09.050943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8cbd4-d22b-4894-9036-7f393bd24432 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:09.052256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8cbd4-d22b-4894-9036-7f393bd24432 service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:09.052758 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8cbd4-d22b-4894-9036-7f393bd24432 service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:09.053213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8cbd4-d22b-4894-9036-7f393bd24432 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:09.062510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f8cbd4-d22b-4894-9036-7f393bd24432 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:09.064169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f8cbd4-d22b-4894-9036-7f393bd24432 service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:09.083178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1114/2217] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:17:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:17:09.625169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73525ea-5715-4aca-8335-838aabc5410e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:09.625471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73525ea-5715-4aca-8335-838aabc5410e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:09.625742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73525ea-5715-4aca-8335-838aabc5410e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:09.646286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-b73525ea-5715-4aca-8335-838aabc5410e None None] MFA Rules not processed for user `db2f1171c5394a1c8879cd09c1e685b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:09.647539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b73525ea-5715-4aca-8335-838aabc5410e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:09.701722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1104/2218] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:17:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:11.239753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8cbd4-d22b-4894-9036-7f393bd24432 service swift] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:11.242957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-15f8cbd4-d22b-4894-9036-7f393bd24432 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:11.308366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8cbd4-d22b-4894-9036-7f393bd24432 service swift] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:11.313916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:11.314554 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:11.314836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:11.315109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] PATH_INFO: `/v3/users/bae66efbdf994be2b6f49ba3e5c08ab6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:11.316285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bae66efbdf994be2b6f49ba3e5c08ab6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:11.318312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:11.368564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bae66efbdf994be2b6f49ba3e5c08ab6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:11.376837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bae66efbdf994be2b6f49ba3e5c08ab6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:11.377462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bae66efbdf994be2b6f49ba3e5c08ab6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:11.378076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bae66efbdf994be2b6f49ba3e5c08ab6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:11.385388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-264a17f4-1b1d-4d61-9508-04aa1f9fd27f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bae66efbdf994be2b6f49ba3e5c08ab6'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:11.403657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1115/2219] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:17:11 2022] DELETE /identity/v3/users/bae66efbdf994be2b6f49ba3e5c08ab6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:17:11.637640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73525ea-5715-4aca-8335-838aabc5410e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:11.651359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73525ea-5715-4aca-8335-838aabc5410e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:11.711563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:11.712223 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:11.712506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:11.713043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] PATH_INFO: `/v3/projects/d14cf31db1244e5d9f5330a55f6dc801` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:11.714173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d14cf31db1244e5d9f5330a55f6dc801)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:11.716197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:11.770459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd14cf31db1244e5d9f5330a55f6dc801', 'request_id': 'req-f26c6086-071c-4041-89d9-96b4f51bec0a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:11.778748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd14cf31db1244e5d9f5330a55f6dc801', 'request_id': 'req-f26c6086-071c-4041-89d9-96b4f51bec0a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:11.785479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd14cf31db1244e5d9f5330a55f6dc801', 'request_id': 'req-f26c6086-071c-4041-89d9-96b4f51bec0a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:11.797731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] The token cache is being invalidate because project d14cf31db1244e5d9f5330a55f6dc801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:17:11.798021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:11.798678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:11.800212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1105/2220] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:17:11 2022] DELETE /identity/v3/projects/d14cf31db1244e5d9f5330a55f6dc801 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:17:13.416463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a07569-13ae-4865-95f6-457b2291234c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:13.416998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a07569-13ae-4865-95f6-457b2291234c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:13.417102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a07569-13ae-4865-95f6-457b2291234c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:13.439539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-08a07569-13ae-4865-95f6-457b2291234c None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:13.441018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-08a07569-13ae-4865-95f6-457b2291234c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:13.502569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1116/2221] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:17:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:13.516500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:13.517949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:13.553302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:17:13.594279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26c6086-071c-4041-89d9-96b4f51bec0a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:13.599574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16acf-a0e9-49cb-adde-dac7bdf91c6d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:13.600173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16acf-a0e9-49cb-adde-dac7bdf91c6d service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:13.600447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16acf-a0e9-49cb-adde-dac7bdf91c6d service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:13.600716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a16acf-a0e9-49cb-adde-dac7bdf91c6d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:13.609029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a16acf-a0e9-49cb-adde-dac7bdf91c6d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:13.610714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a16acf-a0e9-49cb-adde-dac7bdf91c6d service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:13.619373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1106/2222] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:17:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:17:14.223993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77a1e2-86eb-4da6-8046-229dbf9b6982 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:14.224397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77a1e2-86eb-4da6-8046-229dbf9b6982 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:14.224698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb77a1e2-86eb-4da6-8046-229dbf9b6982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:14.244667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-cb77a1e2-86eb-4da6-8046-229dbf9b6982 None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:14.245865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cb77a1e2-86eb-4da6-8046-229dbf9b6982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:14.253027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1117/2223] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 10 14:17:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:14.266280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16acf-a0e9-49cb-adde-dac7bdf91c6d service swift] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:14.276948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a16acf-a0e9-49cb-adde-dac7bdf91c6d service swift] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:14.282681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194e803-6257-4338-8d6e-8d0da366d9d3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:14.283941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194e803-6257-4338-8d6e-8d0da366d9d3 service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:14.284434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194e803-6257-4338-8d6e-8d0da366d9d3 service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:14.285381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194e803-6257-4338-8d6e-8d0da366d9d3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:14.293337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194e803-6257-4338-8d6e-8d0da366d9d3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:14.294935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2194e803-6257-4338-8d6e-8d0da366d9d3 service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:14.303850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1107/2224] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:17:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:17:14.557010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc9483-f8b6-4abb-8cb7-81e20bbbe213 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:14.557522 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc9483-f8b6-4abb-8cb7-81e20bbbe213 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:14.557636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc9483-f8b6-4abb-8cb7-81e20bbbe213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:14.565688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:17:14.578551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-40dc9483-f8b6-4abb-8cb7-81e20bbbe213 None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:14.581473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-40dc9483-f8b6-4abb-8cb7-81e20bbbe213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:14.589438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1118/2225] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:17:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:14.602441 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194e803-6257-4338-8d6e-8d0da366d9d3 service swift] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:14.613219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194e803-6257-4338-8d6e-8d0da366d9d3 service swift] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:14.619548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f499a62-2f07-46ae-aaa7-604e697c848e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:14.620089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f499a62-2f07-46ae-aaa7-604e697c848e service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:14.620368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f499a62-2f07-46ae-aaa7-604e697c848e service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:14.620642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f499a62-2f07-46ae-aaa7-604e697c848e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:14.629311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f499a62-2f07-46ae-aaa7-604e697c848e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:14.631763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f499a62-2f07-46ae-aaa7-604e697c848e service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:14.640569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1108/2226] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:17:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:17:15.061441 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5af7b-db7b-4830-8f1e-9dad2ead9e70 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:15.061727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5af7b-db7b-4830-8f1e-9dad2ead9e70 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:15.062000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5af7b-db7b-4830-8f1e-9dad2ead9e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:15.083195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fdb5af7b-db7b-4830-8f1e-9dad2ead9e70 None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:15.084994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fdb5af7b-db7b-4830-8f1e-9dad2ead9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:15.093014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1119/2227] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 10 14:17:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:15.107726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f499a62-2f07-46ae-aaa7-604e697c848e service swift] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:15.120282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f499a62-2f07-46ae-aaa7-604e697c848e service swift] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:15.126279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44ecd1-1b59-4f15-ba75-c9a451515dd3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:15.126902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ecd1-1b59-4f15-ba75-c9a451515dd3 service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:15.127181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ecd1-1b59-4f15-ba75-c9a451515dd3 service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:15.127471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ecd1-1b59-4f15-ba75-c9a451515dd3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:15.137930 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44ecd1-1b59-4f15-ba75-c9a451515dd3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:15.139561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44ecd1-1b59-4f15-ba75-c9a451515dd3 service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:15.149125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1109/2228] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:17:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:17:17.388175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5af7b-db7b-4830-8f1e-9dad2ead9e70 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:17.389558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fdb5af7b-db7b-4830-8f1e-9dad2ead9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:17.453904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5af7b-db7b-4830-8f1e-9dad2ead9e70 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:17.458651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:17.459213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:17.459515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:17.459813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] PATH_INFO: `/v3/users/ca1929f4d511416a93c784d327691573` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:17.460937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca1929f4d511416a93c784d327691573)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:17.462967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:17.508809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca1929f4d511416a93c784d327691573'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:17.515786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca1929f4d511416a93c784d327691573'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:17.516396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca1929f4d511416a93c784d327691573'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:17.517002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca1929f4d511416a93c784d327691573'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:17.524714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca1929f4d511416a93c784d327691573'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:17.543047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1120/2229] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:17:17 2022] DELETE /identity/v3/users/ca1929f4d511416a93c784d327691573 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:17:17.547457 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab906b3-0893-43c4-86f3-9448f6c7c502 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:17.547940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab906b3-0893-43c4-86f3-9448f6c7c502 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:17.548036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab906b3-0893-43c4-86f3-9448f6c7c502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:17.567397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2ab906b3-0893-43c4-86f3-9448f6c7c502 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:17.568694 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2ab906b3-0893-43c4-86f3-9448f6c7c502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:17.627116 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1110/2230] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:17.637548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:17.639081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:17.698722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1ef89-d15c-4baa-ac02-d6a109cebf6c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:17.702403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88739ba3-e374-4006-86a2-17f3fcdddbd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:17.702998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739ba3-e374-4006-86a2-17f3fcdddbd3 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:17.703288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739ba3-e374-4006-86a2-17f3fcdddbd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:17.703599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88739ba3-e374-4006-86a2-17f3fcdddbd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:17.713708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88739ba3-e374-4006-86a2-17f3fcdddbd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:17.715076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88739ba3-e374-4006-86a2-17f3fcdddbd3 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:17.724966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1121/2231] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:17:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:17:17.893056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab906b3-0893-43c4-86f3-9448f6c7c502 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:17.894565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2ab906b3-0893-43c4-86f3-9448f6c7c502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:17.906652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab906b3-0893-43c4-86f3-9448f6c7c502 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:17.911295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:17.912690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:17.912974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:17.913250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] PATH_INFO: `/v3/projects/37278fff738c474d9090d3324f984711` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:17.914400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37278fff738c474d9090d3324f984711)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:17.916400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:17.964004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37278fff738c474d9090d3324f984711', 'request_id': 'req-f6b87d21-a2ff-41dc-882b-2597af078e49'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:17.971428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37278fff738c474d9090d3324f984711', 'request_id': 'req-f6b87d21-a2ff-41dc-882b-2597af078e49'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:17.977111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37278fff738c474d9090d3324f984711', 'request_id': 'req-f6b87d21-a2ff-41dc-882b-2597af078e49'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:17.988299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] The token cache is being invalidate because project 37278fff738c474d9090d3324f984711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:17:17.988587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:17.989225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6b87d21-a2ff-41dc-882b-2597af078e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:17.991240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1111/2232] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:17:17 2022] DELETE /identity/v3/projects/37278fff738c474d9090d3324f984711 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:17:17.996081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d322222-231a-45a2-a0c3-aadd13b24d28 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:17.996368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d322222-231a-45a2-a0c3-aadd13b24d28 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:17.996638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d322222-231a-45a2-a0c3-aadd13b24d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:18.016368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2d322222-231a-45a2-a0c3-aadd13b24d28 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:18.017549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2d322222-231a-45a2-a0c3-aadd13b24d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:18.026033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:17:18.079685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1122/2233] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:17:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:18.087409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3b8d3-7a83-4087-b7ff-f8fa222e9055 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:18.087848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3b8d3-7a83-4087-b7ff-f8fa222e9055 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:18.088160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea3b8d3-7a83-4087-b7ff-f8fa222e9055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:18.095821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:17:18.109805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-5ea3b8d3-7a83-4087-b7ff-f8fa222e9055 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:18.111008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5ea3b8d3-7a83-4087-b7ff-f8fa222e9055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:18.119463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1112/2234] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:18.123644 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d322222-231a-45a2-a0c3-aadd13b24d28 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:18.134305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d322222-231a-45a2-a0c3-aadd13b24d28 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:18.139178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596085b-e1fb-4894-bbd3-4c11c686a7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:18.139805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596085b-e1fb-4894-bbd3-4c11c686a7b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:18.140078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596085b-e1fb-4894-bbd3-4c11c686a7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:18.140345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7596085b-e1fb-4894-bbd3-4c11c686a7b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:18.140987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596085b-e1fb-4894-bbd3-4c11c686a7b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:17:18.141291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596085b-e1fb-4894-bbd3-4c11c686a7b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:18.142916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7596085b-e1fb-4894-bbd3-4c11c686a7b6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:18.155490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1123/2235] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:17:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:17:18.159726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3b8d3-7a83-4087-b7ff-f8fa222e9055 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:18.169710 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea3b8d3-7a83-4087-b7ff-f8fa222e9055 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:18.174425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96367e93-3daa-4f97-9c65-d256a28cba76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:18.175026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96367e93-3daa-4f97-9c65-d256a28cba76 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:18.175310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96367e93-3daa-4f97-9c65-d256a28cba76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:18.175614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96367e93-3daa-4f97-9c65-d256a28cba76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:18.176249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96367e93-3daa-4f97-9c65-d256a28cba76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:18.178887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96367e93-3daa-4f97-9c65-d256a28cba76 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:18.209783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1113/2236] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:17:18 2022] POST /identity/v3/projects => generated 391 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:17:18.214274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596085b-e1fb-4894-bbd3-4c11c686a7b6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:18.224769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7596085b-e1fb-4894-bbd3-4c11c686a7b6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:18.280692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3539b-c0ae-4da0-98c8-34ba0259440c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:18.281295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3539b-c0ae-4da0-98c8-34ba0259440c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:18.281571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3539b-c0ae-4da0-98c8-34ba0259440c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:18.281842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3539b-c0ae-4da0-98c8-34ba0259440c admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:18.282456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3539b-c0ae-4da0-98c8-34ba0259440c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:18.284415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c3539b-c0ae-4da0-98c8-34ba0259440c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:18.307289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1124/2237] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:17:18 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:17:18.311304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96367e93-3daa-4f97-9c65-d256a28cba76 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:18.321858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96367e93-3daa-4f97-9c65-d256a28cba76 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:18.326480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b855a4-e2fb-47a1-be6e-1365c6deaf03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:18.327111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b855a4-e2fb-47a1-be6e-1365c6deaf03 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:18.327391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b855a4-e2fb-47a1-be6e-1365c6deaf03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:18.327698 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b855a4-e2fb-47a1-be6e-1365c6deaf03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:18.328341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b855a4-e2fb-47a1-be6e-1365c6deaf03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:18.329950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b855a4-e2fb-47a1-be6e-1365c6deaf03 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:18.339992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1114/2238] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:17:18.343848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3539b-c0ae-4da0-98c8-34ba0259440c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:18.354578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3539b-c0ae-4da0-98c8-34ba0259440c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:18.359719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f904b-2a10-404d-999d-6d25352b72b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:18.360310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f904b-2a10-404d-999d-6d25352b72b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:18.360589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f904b-2a10-404d-999d-6d25352b72b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:18.360860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f904b-2a10-404d-999d-6d25352b72b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:18.361499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f904b-2a10-404d-999d-6d25352b72b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:18.363098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f904b-2a10-404d-999d-6d25352b72b3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:18.374465 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1125/2239] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:17:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:17:18.378493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b855a4-e2fb-47a1-be6e-1365c6deaf03 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:18.388713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b855a4-e2fb-47a1-be6e-1365c6deaf03 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:18.393399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32f251-eff2-4e3d-87bc-6ddcc40a89e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:18.394045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32f251-eff2-4e3d-87bc-6ddcc40a89e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:18.394321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32f251-eff2-4e3d-87bc-6ddcc40a89e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:18.394592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32f251-eff2-4e3d-87bc-6ddcc40a89e9 admin admin] PATH_INFO: `/v3/projects/c8237d99cde74759a7d61544895a6c59/users/e25fb4daa7c44af2b4518b53ac2b4fc0/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:18.409434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32f251-eff2-4e3d-87bc-6ddcc40a89e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8237d99cde74759a7d61544895a6c59, user_id=e25fb4daa7c44af2b4518b53ac2b4fc0, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:18.412727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32f251-eff2-4e3d-87bc-6ddcc40a89e9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:18.426217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1115/2240] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:17:18 2022] PUT /identity/v3/projects/c8237d99cde74759a7d61544895a6c59/users/e25fb4daa7c44af2b4518b53ac2b4fc0/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:17:18.440499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55757b84-1ece-4d3c-868a-5c4af5366ba7 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:18.441019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55757b84-1ece-4d3c-868a-5c4af5366ba7 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:18.441626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55757b84-1ece-4d3c-868a-5c4af5366ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:18.469345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-55757b84-1ece-4d3c-868a-5c4af5366ba7 None None] MFA Rules not processed for user `e25fb4daa7c44af2b4518b53ac2b4fc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:18.470491 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-55757b84-1ece-4d3c-868a-5c4af5366ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:18.539784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1126/2241] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:17:18 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:18.554593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32f251-eff2-4e3d-87bc-6ddcc40a89e9 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:18.556351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ae32f251-eff2-4e3d-87bc-6ddcc40a89e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:18.612742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32f251-eff2-4e3d-87bc-6ddcc40a89e9 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:18.616644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f0945e-2051-478b-b72f-a4836f091053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:18.617244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0945e-2051-478b-b72f-a4836f091053 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:18.617516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0945e-2051-478b-b72f-a4836f091053 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:18.617781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f0945e-2051-478b-b72f-a4836f091053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:18.626021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0945e-2051-478b-b72f-a4836f091053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:18.627459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f0945e-2051-478b-b72f-a4836f091053 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:18.636298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1116/2242] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:17:18 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:17:28.535793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:17:28.632379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:17:44.219689 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55757b84-1ece-4d3c-868a-5c4af5366ba7 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:44.282136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55757b84-1ece-4d3c-868a-5c4af5366ba7 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:44.286686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:44.288072 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:44.288406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:44.288668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] PATH_INFO: `/v3/users/e25fb4daa7c44af2b4518b53ac2b4fc0` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:44.289820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e25fb4daa7c44af2b4518b53ac2b4fc0)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:44.291818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:44.339021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e25fb4daa7c44af2b4518b53ac2b4fc0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:44.346336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e25fb4daa7c44af2b4518b53ac2b4fc0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:44.346911 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e25fb4daa7c44af2b4518b53ac2b4fc0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:44.347544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e25fb4daa7c44af2b4518b53ac2b4fc0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:44.354975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e25fb4daa7c44af2b4518b53ac2b4fc0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:44.372505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1127/2243] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:17:44 2022] DELETE /identity/v3/users/e25fb4daa7c44af2b4518b53ac2b4fc0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:17:44.376906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff5331-8809-4f0e-9e29-e80cd614be4c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:44.377406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff5331-8809-4f0e-9e29-e80cd614be4c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:44.377510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ff5331-8809-4f0e-9e29-e80cd614be4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:44.396145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e8ff5331-8809-4f0e-9e29-e80cd614be4c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:17:44.397432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e8ff5331-8809-4f0e-9e29-e80cd614be4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:44.457060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1117/2244] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:17:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:17:44.466755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:44.468211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:44.533248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf746d47-17bf-43ab-8396-72964cb02ad2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:44.536963 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bcd05-53aa-41be-9771-f432ceb8679c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:44.538129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bcd05-53aa-41be-9771-f432ceb8679c service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:44.538412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bcd05-53aa-41be-9771-f432ceb8679c service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:44.538708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bcd05-53aa-41be-9771-f432ceb8679c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:44.547952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bcd05-53aa-41be-9771-f432ceb8679c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:44.549384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bcd05-53aa-41be-9771-f432ceb8679c service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:44.559674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1128/2245] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:17:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:17:44.912443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff5331-8809-4f0e-9e29-e80cd614be4c None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:17:44.914027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e8ff5331-8809-4f0e-9e29-e80cd614be4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:17:44.929894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ff5331-8809-4f0e-9e29-e80cd614be4c None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:17:44.935251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:17:44.935947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:17:44.936224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:17:44.936501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] PATH_INFO: `/v3/projects/c8237d99cde74759a7d61544895a6c59` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:17:44.937665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8237d99cde74759a7d61544895a6c59)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:17:44.939628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:17:45.001813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8237d99cde74759a7d61544895a6c59', 'request_id': 'req-752a055f-5a3f-4851-b8e4-c37b15ce5843'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:45.010983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8237d99cde74759a7d61544895a6c59', 'request_id': 'req-752a055f-5a3f-4851-b8e4-c37b15ce5843'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:45.019524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8237d99cde74759a7d61544895a6c59', 'request_id': 'req-752a055f-5a3f-4851-b8e4-c37b15ce5843'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:45.032909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] The token cache is being invalidate because project c8237d99cde74759a7d61544895a6c59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:17:45.033195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:45.033987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:17:45.035640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1118/2246] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:17:44 2022] DELETE /identity/v3/projects/c8237d99cde74759a7d61544895a6c59 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:17:54.555382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:17:55.029590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:18:09.991527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdf456-882b-4e3a-8930-eb354fd1cd7a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:09.992173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdf456-882b-4e3a-8930-eb354fd1cd7a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:09.992279 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fdf456-882b-4e3a-8930-eb354fd1cd7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:10.012743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-53fdf456-882b-4e3a-8930-eb354fd1cd7a None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:10.013873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-53fdf456-882b-4e3a-8930-eb354fd1cd7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:10.070846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1129/2247] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:18:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:18:10.094058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:10.095608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:10.167960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a055f-5a3f-4851-b8e4-c37b15ce5843 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:10.172961 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2bc1f-6029-4b16-82c1-17a6e68fa562 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:10.173504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2bc1f-6029-4b16-82c1-17a6e68fa562 service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:10.173920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2bc1f-6029-4b16-82c1-17a6e68fa562 service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:10.174208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2bc1f-6029-4b16-82c1-17a6e68fa562 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:10.182767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2bc1f-6029-4b16-82c1-17a6e68fa562 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:10.184450 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2bc1f-6029-4b16-82c1-17a6e68fa562 service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:10.192667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1119/2248] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:18:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:18:20.063259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=25 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:18:20.189887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=32 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:18:29.685000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b12adb-7634-4c02-afbb-648ca3078aaf None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:29.685530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b12adb-7634-4c02-afbb-648ca3078aaf None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:29.685530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b12adb-7634-4c02-afbb-648ca3078aaf None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:29.686934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1130/2249] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 10 14:18:29.695575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d00aea-0e29-473d-8ca1-7f56887beb66 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:29.696190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d00aea-0e29-473d-8ca1-7f56887beb66 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:29.696377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d00aea-0e29-473d-8ca1-7f56887beb66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:29.717518 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c6d00aea-0e29-473d-8ca1-7f56887beb66 None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:29.718719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c6d00aea-0e29-473d-8ca1-7f56887beb66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:29.793693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1120/2250] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:18:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:18:29.806220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b12adb-7634-4c02-afbb-648ca3078aaf None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:29.807864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-63b12adb-7634-4c02-afbb-648ca3078aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:29.870524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b12adb-7634-4c02-afbb-648ca3078aaf None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:29.874704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18424122-8dcd-45dc-ba3e-904135cf6628 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf75b583add44b6b87bff5dc67080d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:29.875304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18424122-8dcd-45dc-ba3e-904135cf6628 service glance] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:29.875716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18424122-8dcd-45dc-ba3e-904135cf6628 service glance] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:29.875987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18424122-8dcd-45dc-ba3e-904135cf6628 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:29.886966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18424122-8dcd-45dc-ba3e-904135cf6628 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:29.888443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18424122-8dcd-45dc-ba3e-904135cf6628 service glance] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:29.900478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1131/2251] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:18:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:18:29.967797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b93feb-d720-44cf-a052-2230fbc6f4bf None None] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:29.968071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b93feb-d720-44cf-a052-2230fbc6f4bf None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:29.968338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b93feb-d720-44cf-a052-2230fbc6f4bf None None] PATH_INFO: `/` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:29.969677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1121/2252] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:18:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:18:29.974538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b5b3-07ae-471b-87ef-8d1a86195fa1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:29.974819 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b5b3-07ae-471b-87ef-8d1a86195fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:29.975093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e9b5b3-07ae-471b-87ef-8d1a86195fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:29.996599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-01e9b5b3-07ae-471b-87ef-8d1a86195fa1 None None] MFA Rules not processed for user `12c71caf31354ec193e3e2da231f1e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:29.997803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-01e9b5b3-07ae-471b-87ef-8d1a86195fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:30.052041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1132/2253] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:18:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:18:30.063762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b93feb-d720-44cf-a052-2230fbc6f4bf None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:30.065466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a0b93feb-d720-44cf-a052-2230fbc6f4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:30.079137 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b93feb-d720-44cf-a052-2230fbc6f4bf None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:30.082654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe148b99-0d99-42d7-b9a1-5af0778e2998 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:30.083242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe148b99-0d99-42d7-b9a1-5af0778e2998 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:30.083554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe148b99-0d99-42d7-b9a1-5af0778e2998 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:30.083828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe148b99-0d99-42d7-b9a1-5af0778e2998 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:30.091349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe148b99-0d99-42d7-b9a1-5af0778e2998 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:30.092801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe148b99-0d99-42d7-b9a1-5af0778e2998 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:30.100648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1122/2254] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:18:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:18:30.182923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9396630-3e7a-4cef-a144-618792fb77bb None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:30.183214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9396630-3e7a-4cef-a144-618792fb77bb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:30.183514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9396630-3e7a-4cef-a144-618792fb77bb None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:30.184893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1133/2255] 10.222.0.26 () {54 vars in 921 bytes} [Sat Sep 10 14:18:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:18:30.189731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26121ed-f33c-497e-9174-99adfab2b379 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:30.190015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26121ed-f33c-497e-9174-99adfab2b379 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:30.190286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26121ed-f33c-497e-9174-99adfab2b379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:30.210822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e26121ed-f33c-497e-9174-99adfab2b379 None None] MFA Rules not processed for user `cb24a1b55ee7408ab24c68991aec54ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:30.212065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e26121ed-f33c-497e-9174-99adfab2b379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:30.221343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1123/2256] 10.222.0.26 () {58 vars in 1018 bytes} [Sat Sep 10 14:18:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:18:30.231914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9396630-3e7a-4cef-a144-618792fb77bb None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:30.243150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9396630-3e7a-4cef-a144-618792fb77bb None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:30.246691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14215f99-39e6-4023-b7a3-95da16bafc26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:30.247272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14215f99-39e6-4023-b7a3-95da16bafc26 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:30.247576 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14215f99-39e6-4023-b7a3-95da16bafc26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:30.247842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14215f99-39e6-4023-b7a3-95da16bafc26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:30.259182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14215f99-39e6-4023-b7a3-95da16bafc26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:30.260640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14215f99-39e6-4023-b7a3-95da16bafc26 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:30.271089 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1134/2257] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:18:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:18:32.094418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26121ed-f33c-497e-9174-99adfab2b379 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:32.095875 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e26121ed-f33c-497e-9174-99adfab2b379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:32.107277 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26121ed-f33c-497e-9174-99adfab2b379 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:32.111192 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8619a658-7722-43a8-862a-c9cc365952eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:32.111824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619a658-7722-43a8-862a-c9cc365952eb service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:32.112102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619a658-7722-43a8-862a-c9cc365952eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:32.112393 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8619a658-7722-43a8-862a-c9cc365952eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:32.114103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-8619a658-7722-43a8-862a-c9cc365952eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:32.128132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619a658-7722-43a8-862a-c9cc365952eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:32.129562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8619a658-7722-43a8-862a-c9cc365952eb service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:32.141611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1124/2258] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:18:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:18:33.329261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14215f99-39e6-4023-b7a3-95da16bafc26 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:33.337995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14215f99-39e6-4023-b7a3-95da16bafc26 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:33.341606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0449d-0987-4751-810f-13463d35879e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:33.342198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0449d-0987-4751-810f-13463d35879e service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:33.342473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0449d-0987-4751-810f-13463d35879e service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:33.342743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0449d-0987-4751-810f-13463d35879e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:33.344480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8ca0449d-0987-4751-810f-13463d35879e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:33.364806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0449d-0987-4751-810f-13463d35879e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:33.366320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0449d-0987-4751-810f-13463d35879e service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:33.379954 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1135/2259] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:18:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:18:33.572768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392e7a2-07d8-4099-b341-e3f4ed7a7b99 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:33.573250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392e7a2-07d8-4099-b341-e3f4ed7a7b99 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:33.573363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1392e7a2-07d8-4099-b341-e3f4ed7a7b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:33.592653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-1392e7a2-07d8-4099-b341-e3f4ed7a7b99 None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:33.593844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1392e7a2-07d8-4099-b341-e3f4ed7a7b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:33.601634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1125/2260] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:18:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:18:33.613684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0449d-0987-4751-810f-13463d35879e service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:33.625282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0449d-0987-4751-810f-13463d35879e service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:33.630210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299152a1-76f1-4847-ac14-0d408e5e22a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:33.630795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299152a1-76f1-4847-ac14-0d408e5e22a7 service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:33.631069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299152a1-76f1-4847-ac14-0d408e5e22a7 service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:33.631337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299152a1-76f1-4847-ac14-0d408e5e22a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:33.639368 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299152a1-76f1-4847-ac14-0d408e5e22a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:33.640965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299152a1-76f1-4847-ac14-0d408e5e22a7 service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:33.649393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1136/2261] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:18:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:18:39.424399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392e7a2-07d8-4099-b341-e3f4ed7a7b99 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:39.427471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1392e7a2-07d8-4099-b341-e3f4ed7a7b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:39.508736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1392e7a2-07d8-4099-b341-e3f4ed7a7b99 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:39.513816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:39.514455 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:39.514733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:39.515001 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] PATH_INFO: `/v3/users/8f468d0b1fec4502858b8e29ac640ebd` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:39.516261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f468d0b1fec4502858b8e29ac640ebd)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:39.518391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:39.567285 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f468d0b1fec4502858b8e29ac640ebd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:18:39.567955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f468d0b1fec4502858b8e29ac640ebd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:18:39.575425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f468d0b1fec4502858b8e29ac640ebd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:18:39.576719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:18:39.588479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f468d0b1fec4502858b8e29ac640ebd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:18:39.589462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-556c7f48-65ec-47dd-913c-be7faaadbd51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f468d0b1fec4502858b8e29ac640ebd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:18:39.608385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1126/2262] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:18:39 2022] DELETE /identity/v3/users/8f468d0b1fec4502858b8e29ac640ebd => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:18:39.790351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299152a1-76f1-4847-ac14-0d408e5e22a7 service swift] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:39.812467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299152a1-76f1-4847-ac14-0d408e5e22a7 service swift] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:39.884110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:39.884747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:39.885029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:39.885301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] PATH_INFO: `/v3/projects/e3a9c95bc33f4697917a9e8c53f78a98` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:39.886420 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3a9c95bc33f4697917a9e8c53f78a98)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:39.888436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:39.907287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:18:39.945218 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3a9c95bc33f4697917a9e8c53f78a98', 'request_id': 'req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:18:39.950835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3a9c95bc33f4697917a9e8c53f78a98', 'request_id': 'req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:18:39.964386 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3a9c95bc33f4697917a9e8c53f78a98', 'request_id': 'req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:18:39.977574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] The token cache is being invalidate because project e3a9c95bc33f4697917a9e8c53f78a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:18:39.977875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:18:39.978549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-afb7b4ac-d3ae-4f20-aa32-6b8fa72d96a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:18:39.983478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1137/2263] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:18:39 2022] DELETE /identity/v3/projects/e3a9c95bc33f4697917a9e8c53f78a98 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:18:39.985528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb31f1-b0f6-4f51-b1ac-4cea4df6101c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:39.985844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb31f1-b0f6-4f51-b1ac-4cea4df6101c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:39.986123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb31f1-b0f6-4f51-b1ac-4cea4df6101c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:40.009291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-76eb31f1-b0f6-4f51-b1ac-4cea4df6101c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:40.010483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-76eb31f1-b0f6-4f51-b1ac-4cea4df6101c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:40.072788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1127/2264] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:18:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:18:40.081543 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab6049-875e-4cde-894f-4fc8850ebc66 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:40.081852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab6049-875e-4cde-894f-4fc8850ebc66 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:40.082127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab6049-875e-4cde-894f-4fc8850ebc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:40.103892 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-81ab6049-875e-4cde-894f-4fc8850ebc66 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:40.105119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-81ab6049-875e-4cde-894f-4fc8850ebc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:40.113841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1138/2265] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:18:40.118432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb31f1-b0f6-4f51-b1ac-4cea4df6101c None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:40.128707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb31f1-b0f6-4f51-b1ac-4cea4df6101c None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:40.133956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb084766-9c58-4b47-9445-4c471322382f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:40.134568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb084766-9c58-4b47-9445-4c471322382f admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:40.134844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb084766-9c58-4b47-9445-4c471322382f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:40.135114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb084766-9c58-4b47-9445-4c471322382f admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:40.135815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb084766-9c58-4b47-9445-4c471322382f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:18:40.136114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb084766-9c58-4b47-9445-4c471322382f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:40.137780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb084766-9c58-4b47-9445-4c471322382f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:40.149606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1128/2266] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:18:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:18:40.154290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab6049-875e-4cde-894f-4fc8850ebc66 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:40.165816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab6049-875e-4cde-894f-4fc8850ebc66 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:40.170514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc61ce-d06c-4913-9ce1-216db36d468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:40.171115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc61ce-d06c-4913-9ce1-216db36d468f admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:40.171423 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc61ce-d06c-4913-9ce1-216db36d468f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:40.171729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc61ce-d06c-4913-9ce1-216db36d468f admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:40.172369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc61ce-d06c-4913-9ce1-216db36d468f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:40.174343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc61ce-d06c-4913-9ce1-216db36d468f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:40.205056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1139/2267] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:18:40 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:18:40.209062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb084766-9c58-4b47-9445-4c471322382f admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:40.219769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb084766-9c58-4b47-9445-4c471322382f admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:40.282388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac99f51-a9bd-44c7-b38d-97071366692e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:40.283133 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac99f51-a9bd-44c7-b38d-97071366692e admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:40.283512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac99f51-a9bd-44c7-b38d-97071366692e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:40.283789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac99f51-a9bd-44c7-b38d-97071366692e admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:40.284434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac99f51-a9bd-44c7-b38d-97071366692e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:40.286363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac99f51-a9bd-44c7-b38d-97071366692e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:40.309335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1129/2268] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:18:40 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:18:40.313499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc61ce-d06c-4913-9ce1-216db36d468f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:40.324098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc61ce-d06c-4913-9ce1-216db36d468f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:40.329171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553de878-8f47-4b6f-8296-39312b348cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:40.329771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553de878-8f47-4b6f-8296-39312b348cd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:40.330059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553de878-8f47-4b6f-8296-39312b348cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:40.330324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553de878-8f47-4b6f-8296-39312b348cd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:40.330992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553de878-8f47-4b6f-8296-39312b348cd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:40.332754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553de878-8f47-4b6f-8296-39312b348cd1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:40.344854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1140/2269] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:18:40.348419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac99f51-a9bd-44c7-b38d-97071366692e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:40.361376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac99f51-a9bd-44c7-b38d-97071366692e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:40.366062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30277d0-7bf2-46cd-9e5d-2d631733cbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:40.367307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30277d0-7bf2-46cd-9e5d-2d631733cbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:40.367868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30277d0-7bf2-46cd-9e5d-2d631733cbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:40.368146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30277d0-7bf2-46cd-9e5d-2d631733cbf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:40.368787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30277d0-7bf2-46cd-9e5d-2d631733cbf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:40.370372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30277d0-7bf2-46cd-9e5d-2d631733cbf2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:40.381674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1130/2270] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:18:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:18:40.386391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553de878-8f47-4b6f-8296-39312b348cd1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:40.397386 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553de878-8f47-4b6f-8296-39312b348cd1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:40.403234 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e4adb-222c-4130-85d4-4d022ed3ed4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:40.403938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e4adb-222c-4130-85d4-4d022ed3ed4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:40.404216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e4adb-222c-4130-85d4-4d022ed3ed4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:40.404489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627e4adb-222c-4130-85d4-4d022ed3ed4a admin admin] PATH_INFO: `/v3/projects/6109b45e7ff14addb54c4bff4fb43df6/users/993bf8937c8349e7887b3825a7034367/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:40.423252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e4adb-222c-4130-85d4-4d022ed3ed4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6109b45e7ff14addb54c4bff4fb43df6, user_id=993bf8937c8349e7887b3825a7034367, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:40.425996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627e4adb-222c-4130-85d4-4d022ed3ed4a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:40.440431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1141/2271] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:18:40 2022] PUT /identity/v3/projects/6109b45e7ff14addb54c4bff4fb43df6/users/993bf8937c8349e7887b3825a7034367/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:18:40.445793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3808b-deac-42a6-bd79-92c3da4153d5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:40.446256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3808b-deac-42a6-bd79-92c3da4153d5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:40.446349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3808b-deac-42a6-bd79-92c3da4153d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:40.466670 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-73f3808b-deac-42a6-bd79-92c3da4153d5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:40.468015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-73f3808b-deac-42a6-bd79-92c3da4153d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:40.507548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:18:40.537119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1131/2272] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:18:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:18:40.548082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e4adb-222c-4130-85d4-4d022ed3ed4a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:40.549914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-627e4adb-222c-4130-85d4-4d022ed3ed4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:40.618707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627e4adb-222c-4130-85d4-4d022ed3ed4a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:40.622999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b622642-40a7-44aa-a46d-37f4788ab963 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:40.623688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622642-40a7-44aa-a46d-37f4788ab963 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:40.623965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622642-40a7-44aa-a46d-37f4788ab963 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:40.624289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b622642-40a7-44aa-a46d-37f4788ab963 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:40.637840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b622642-40a7-44aa-a46d-37f4788ab963 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:40.639399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b622642-40a7-44aa-a46d-37f4788ab963 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:40.650215 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1142/2273] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:18:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:18:45.611607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f5ce5-3325-4ad1-9970-0ef9944e1cea None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:45.612572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f5ce5-3325-4ad1-9970-0ef9944e1cea None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:45.613009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f5ce5-3325-4ad1-9970-0ef9944e1cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:45.644225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-101f5ce5-3325-4ad1-9970-0ef9944e1cea None None] MFA Rules not processed for user `993bf8937c8349e7887b3825a7034367`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:45.645741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-101f5ce5-3325-4ad1-9970-0ef9944e1cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:45.736964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1132/2274] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:18:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:18:45.741367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b622642-40a7-44aa-a46d-37f4788ab963 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:45.751645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b622642-40a7-44aa-a46d-37f4788ab963 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:45.757256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccb81e-93c6-4374-935b-cfc91c677eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:45.757866 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccb81e-93c6-4374-935b-cfc91c677eab admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:45.758143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccb81e-93c6-4374-935b-cfc91c677eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:45.758414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccb81e-93c6-4374-935b-cfc91c677eab admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:45.759055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fccb81e-93c6-4374-935b-cfc91c677eab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:45.761134 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fccb81e-93c6-4374-935b-cfc91c677eab admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:45.797912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1143/2275] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:18:45 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:18:45.802967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f5ce5-3325-4ad1-9970-0ef9944e1cea None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:45.820735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101f5ce5-3325-4ad1-9970-0ef9944e1cea None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:45.895089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e1e47-b13f-4e1c-919d-5a3eee842e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:45.896089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e1e47-b13f-4e1c-919d-5a3eee842e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:45.896540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e1e47-b13f-4e1c-919d-5a3eee842e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:45.896977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e1e47-b13f-4e1c-919d-5a3eee842e8d admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:45.897946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e1e47-b13f-4e1c-919d-5a3eee842e8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:45.901127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e1e47-b13f-4e1c-919d-5a3eee842e8d admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:45.949536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccb81e-93c6-4374-935b-cfc91c677eab admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:45.950701 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1133/2276] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:18:45 2022] POST /identity/v3/users => generated 298 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:18:45.962254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccb81e-93c6-4374-935b-cfc91c677eab admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:45.967697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931d340-7269-45f0-b5f8-51ca02223b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:45.968763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931d340-7269-45f0-b5f8-51ca02223b58 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:45.969084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931d340-7269-45f0-b5f8-51ca02223b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:45.969365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931d340-7269-45f0-b5f8-51ca02223b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:45.970027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931d340-7269-45f0-b5f8-51ca02223b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:45.971983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c931d340-7269-45f0-b5f8-51ca02223b58 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:45.993552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e1e47-b13f-4e1c-919d-5a3eee842e8d admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:45.995556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1144/2277] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:18:45 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:18:46.005235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e1e47-b13f-4e1c-919d-5a3eee842e8d admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:46.011101 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42934b11-cee1-42b1-816c-1d3e8a26daed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:46.011987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42934b11-cee1-42b1-816c-1d3e8a26daed admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:46.012368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42934b11-cee1-42b1-816c-1d3e8a26daed admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:46.012943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42934b11-cee1-42b1-816c-1d3e8a26daed admin admin] PATH_INFO: `/v3/projects/4777c5b655354f59a160176da328c8ee/users/059c8b51434f43f7a3e85581094c9a02/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:46.036544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42934b11-cee1-42b1-816c-1d3e8a26daed admin admin] RBAC: Authorizing `identity:create_grant(project_id=4777c5b655354f59a160176da328c8ee, user_id=059c8b51434f43f7a3e85581094c9a02, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:46.040306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42934b11-cee1-42b1-816c-1d3e8a26daed admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:46.062187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1134/2278] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:18:45 2022] PUT /identity/v3/projects/4777c5b655354f59a160176da328c8ee/users/059c8b51434f43f7a3e85581094c9a02/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:18:46.066654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931d340-7269-45f0-b5f8-51ca02223b58 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:46.080524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931d340-7269-45f0-b5f8-51ca02223b58 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:46.085902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:18:46.147494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5d45ab-9cca-4e01-8d3f-4d41874433aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:46.148298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d45ab-9cca-4e01-8d3f-4d41874433aa admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:46.148573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d45ab-9cca-4e01-8d3f-4d41874433aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:46.148843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5d45ab-9cca-4e01-8d3f-4d41874433aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:46.150227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d45ab-9cca-4e01-8d3f-4d41874433aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:46.152951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5d45ab-9cca-4e01-8d3f-4d41874433aa admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:46.165563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1145/2279] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:18:46.171963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42934b11-cee1-42b1-816c-1d3e8a26daed admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:46.184441 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42934b11-cee1-42b1-816c-1d3e8a26daed admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:46.189910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c75a95-a169-4253-9e3c-97f463a56f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:46.191043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c75a95-a169-4253-9e3c-97f463a56f76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:46.191339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c75a95-a169-4253-9e3c-97f463a56f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:46.191659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c75a95-a169-4253-9e3c-97f463a56f76 admin admin] PATH_INFO: `/v3/projects/4777c5b655354f59a160176da328c8ee/users/059c8b51434f43f7a3e85581094c9a02/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:46.194039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c75a95-a169-4253-9e3c-97f463a56f76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4777c5b655354f59a160176da328c8ee, user_id=059c8b51434f43f7a3e85581094c9a02, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:46.197073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c75a95-a169-4253-9e3c-97f463a56f76 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:46.237623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1135/2280] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:18:46 2022] PUT /identity/v3/projects/4777c5b655354f59a160176da328c8ee/users/059c8b51434f43f7a3e85581094c9a02/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:18:50.896660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc349b-bb93-4868-83ca-e649f654b0dc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:50.897793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc349b-bb93-4868-83ca-e649f654b0dc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:50.897793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc349b-bb93-4868-83ca-e649f654b0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:50.927163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-83bc349b-bb93-4868-83ca-e649f654b0dc None None] MFA Rules not processed for user `059c8b51434f43f7a3e85581094c9a02`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:50.928348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-83bc349b-bb93-4868-83ca-e649f654b0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:51.021989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1146/2281] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:18:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:18:51.040131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c75a95-a169-4253-9e3c-97f463a56f76 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:51.042061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f3c75a95-a169-4253-9e3c-97f463a56f76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:51.071634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:18:51.079183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc349b-bb93-4868-83ca-e649f654b0dc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:51.126640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c75a95-a169-4253-9e3c-97f463a56f76 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:51.130416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3172e-2104-4129-a894-c0c524da3398 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf75b583add44b6b87bff5dc67080d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:51.131290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3172e-2104-4129-a894-c0c524da3398 service glance] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:51.132173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3172e-2104-4129-a894-c0c524da3398 service glance] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:51.132462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3172e-2104-4129-a894-c0c524da3398 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:51.156925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc349b-bb93-4868-83ca-e649f654b0dc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:51.162511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8499d1-9037-44b4-94df-f1cce04dc165 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:51.163686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8499d1-9037-44b4-94df-f1cce04dc165 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:51.163970 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8499d1-9037-44b4-94df-f1cce04dc165 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:51.164243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8499d1-9037-44b4-94df-f1cce04dc165 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:51.168226 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9e8499d1-9037-44b4-94df-f1cce04dc165 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:51.204887 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3172e-2104-4129-a894-c0c524da3398 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:51.206436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f3172e-2104-4129-a894-c0c524da3398 service glance] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:51.217470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1136/2282] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:18:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:18:51.234031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:18:51.280752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8499d1-9037-44b4-94df-f1cce04dc165 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:51.282469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8499d1-9037-44b4-94df-f1cce04dc165 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:51.293421 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1147/2283] 10.222.0.26 () {60 vars in 1396 bytes} [Sat Sep 10 14:18:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3266 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:18:51.400637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9681e-350f-45d8-a6dc-2c575dbfcf78 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:51.400920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9681e-350f-45d8-a6dc-2c575dbfcf78 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:51.401195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a9681e-350f-45d8-a6dc-2c575dbfcf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:51.422022 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-70a9681e-350f-45d8-a6dc-2c575dbfcf78 None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:18:51.423234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-70a9681e-350f-45d8-a6dc-2c575dbfcf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:51.484871 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1137/2284] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:18:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:18:51.498823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8499d1-9037-44b4-94df-f1cce04dc165 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:18:51.500298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9e8499d1-9037-44b4-94df-f1cce04dc165 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:18:51.575732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8499d1-9037-44b4-94df-f1cce04dc165 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:18:51.580609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98eff0-7325-43d5-a7f1-68a7956a57da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:18:51.581212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98eff0-7325-43d5-a7f1-68a7956a57da service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:18:51.581484 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98eff0-7325-43d5-a7f1-68a7956a57da service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:18:51.581752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98eff0-7325-43d5-a7f1-68a7956a57da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:18:51.591673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98eff0-7325-43d5-a7f1-68a7956a57da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:18:51.593291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98eff0-7325-43d5-a7f1-68a7956a57da service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:18:51.602482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1148/2285] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:18:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:19:00.685446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca736-0ed6-4be3-825b-71897c03cc4f None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:00.686018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca736-0ed6-4be3-825b-71897c03cc4f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:00.686018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bca736-0ed6-4be3-825b-71897c03cc4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:00.705665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-f4bca736-0ed6-4be3-825b-71897c03cc4f None None] MFA Rules not processed for user `993bf8937c8349e7887b3825a7034367`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:19:00.706951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f4bca736-0ed6-4be3-825b-71897c03cc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:00.714059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1138/2286] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:19:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:19:00.981155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98eff0-7325-43d5-a7f1-68a7956a57da service swift] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:00.982684 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-af98eff0-7325-43d5-a7f1-68a7956a57da service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:00.995753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98eff0-7325-43d5-a7f1-68a7956a57da service swift] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:00.999256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda363d1-00ce-4c16-b921-0c6713216f64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:00.999877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda363d1-00ce-4c16-b921-0c6713216f64 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:01.000150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda363d1-00ce-4c16-b921-0c6713216f64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:01.000415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda363d1-00ce-4c16-b921-0c6713216f64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:01.002249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-bda363d1-00ce-4c16-b921-0c6713216f64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:01.012403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda363d1-00ce-4c16-b921-0c6713216f64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:01.013832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda363d1-00ce-4c16-b921-0c6713216f64 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:01.021657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1149/2287] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:19:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:19:01.343437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bca736-0ed6-4be3-825b-71897c03cc4f None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:01.345263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f4bca736-0ed6-4be3-825b-71897c03cc4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:01.421586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bca736-0ed6-4be3-825b-71897c03cc4f None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:01.426684 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62da61-116b-4fc7-9939-b244876cc00b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:01.427303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62da61-116b-4fc7-9939-b244876cc00b service placement] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:01.427611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62da61-116b-4fc7-9939-b244876cc00b service placement] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:01.427883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf62da61-116b-4fc7-9939-b244876cc00b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:01.429781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-cf62da61-116b-4fc7-9939-b244876cc00b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:01.445542 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62da61-116b-4fc7-9939-b244876cc00b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:01.447160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf62da61-116b-4fc7-9939-b244876cc00b service placement] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:01.457335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1139/2288] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:19:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:19:03.527557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1756e-0d38-4974-befd-f9943bb01a6d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:03.529232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1756e-0d38-4974-befd-f9943bb01a6d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:03.529514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f1756e-0d38-4974-befd-f9943bb01a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:03.553687 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-89f1756e-0d38-4974-befd-f9943bb01a6d None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:19:03.555867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-89f1756e-0d38-4974-befd-f9943bb01a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:03.570680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1150/2289] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:19:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:19:03.595722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62da61-116b-4fc7-9939-b244876cc00b service placement] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:03.595722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:19:03.606889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf62da61-116b-4fc7-9939-b244876cc00b service placement] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:03.612185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd793377-1174-4c75-a8f7-b61d6f4aa6ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:03.612788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd793377-1174-4c75-a8f7-b61d6f4aa6ee service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:03.613062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd793377-1174-4c75-a8f7-b61d6f4aa6ee service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:03.613328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd793377-1174-4c75-a8f7-b61d6f4aa6ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:03.626314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd793377-1174-4c75-a8f7-b61d6f4aa6ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:03.628124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd793377-1174-4c75-a8f7-b61d6f4aa6ee service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:03.641901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1140/2290] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:19:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:19:13.546802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=48 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:19:13.634515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:19:19.040016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1756e-0d38-4974-befd-f9943bb01a6d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:19.050298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f1756e-0d38-4974-befd-f9943bb01a6d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:19.053873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23618e98-dfc4-4201-a017-aa22a4b92b55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:19.054481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23618e98-dfc4-4201-a017-aa22a4b92b55 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:19.054734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23618e98-dfc4-4201-a017-aa22a4b92b55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:19.055001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23618e98-dfc4-4201-a017-aa22a4b92b55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:19.056741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-23618e98-dfc4-4201-a017-aa22a4b92b55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:19.066654 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23618e98-dfc4-4201-a017-aa22a4b92b55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:19.068119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23618e98-dfc4-4201-a017-aa22a4b92b55 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:19.076774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1151/2291] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:19:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:19:19.585351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aad18-26f9-4ed3-a970-73c68f2cc626 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:19.586383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aad18-26f9-4ed3-a970-73c68f2cc626 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:19.586660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aad18-26f9-4ed3-a970-73c68f2cc626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:19.607883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-0c8aad18-26f9-4ed3-a970-73c68f2cc626 None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:19:19.609104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0c8aad18-26f9-4ed3-a970-73c68f2cc626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:19.616516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1141/2292] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 10 14:19:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:19:19.640719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23618e98-dfc4-4201-a017-aa22a4b92b55 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:19.653756 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23618e98-dfc4-4201-a017-aa22a4b92b55 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:19.658787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7dd86-0020-49b1-b018-094d7e3c1b91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:19.659315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7dd86-0020-49b1-b018-094d7e3c1b91 service swift] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:19.659619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7dd86-0020-49b1-b018-094d7e3c1b91 service swift] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:19.659889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca7dd86-0020-49b1-b018-094d7e3c1b91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:19.667823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7dd86-0020-49b1-b018-094d7e3c1b91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:19.669398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca7dd86-0020-49b1-b018-094d7e3c1b91 service swift] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:19.677270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1152/2293] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:19:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:19:27.002022 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8aad18-26f9-4ed3-a970-73c68f2cc626 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:27.003523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0c8aad18-26f9-4ed3-a970-73c68f2cc626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:27.077515 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8aad18-26f9-4ed3-a970-73c68f2cc626 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:27.082826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73031f96-981f-469f-b19a-697a2bdec929 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:27.083425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73031f96-981f-469f-b19a-697a2bdec929 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:27.083729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73031f96-981f-469f-b19a-697a2bdec929 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:27.083999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73031f96-981f-469f-b19a-697a2bdec929 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:27.085700 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-73031f96-981f-469f-b19a-697a2bdec929 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:27.097484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73031f96-981f-469f-b19a-697a2bdec929 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:27.098989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73031f96-981f-469f-b19a-697a2bdec929 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:27.110370 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1142/2294] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:19:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:19:27.983979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7dd86-0020-49b1-b018-094d7e3c1b91 service swift] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:27.996324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca7dd86-0020-49b1-b018-094d7e3c1b91 service swift] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:28.001721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559d47a-1386-4ae1-b4d8-4f3ccb0a0204 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:28.002305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559d47a-1386-4ae1-b4d8-4f3ccb0a0204 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:28.002584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559d47a-1386-4ae1-b4d8-4f3ccb0a0204 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:28.002850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559d47a-1386-4ae1-b4d8-4f3ccb0a0204 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:28.004584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c559d47a-1386-4ae1-b4d8-4f3ccb0a0204 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:28.062246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559d47a-1386-4ae1-b4d8-4f3ccb0a0204 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:28.063880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c559d47a-1386-4ae1-b4d8-4f3ccb0a0204 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:28.073555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1153/2295] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:19:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:19:37.107057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=37 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:19:38.068803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=40 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:19:38.858996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73031f96-981f-469f-b19a-697a2bdec929 service nova] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:38.860593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-73031f96-981f-469f-b19a-697a2bdec929 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:38.874593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73031f96-981f-469f-b19a-697a2bdec929 service nova] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:38.880312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41178373-60f8-4503-b07c-5a558d874400 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:38.880910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41178373-60f8-4503-b07c-5a558d874400 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:38.881189 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41178373-60f8-4503-b07c-5a558d874400 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:38.881460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41178373-60f8-4503-b07c-5a558d874400 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:38.883256 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-41178373-60f8-4503-b07c-5a558d874400 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:38.942777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41178373-60f8-4503-b07c-5a558d874400 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:38.944415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41178373-60f8-4503-b07c-5a558d874400 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:38.953914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1143/2296] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:19:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:19:44.009755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559d47a-1386-4ae1-b4d8-4f3ccb0a0204 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:44.021455 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559d47a-1386-4ae1-b4d8-4f3ccb0a0204 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:44.026964 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e4bc4-978f-43ee-a918-65c1a3598a84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:44.027590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4bc4-978f-43ee-a918-65c1a3598a84 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:44.027875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4bc4-978f-43ee-a918-65c1a3598a84 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:44.028676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5e4bc4-978f-43ee-a918-65c1a3598a84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:44.030637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1a5e4bc4-978f-43ee-a918-65c1a3598a84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:44.088765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e4bc4-978f-43ee-a918-65c1a3598a84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:44.090910 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5e4bc4-978f-43ee-a918-65c1a3598a84 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:44.100457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1154/2297] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:19:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:19:45.370947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41178373-60f8-4503-b07c-5a558d874400 service nova] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:45.380956 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41178373-60f8-4503-b07c-5a558d874400 service nova] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:45.384552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ec464-f123-4bdf-96ca-dd8d2d5e40d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:45.385225 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ec464-f123-4bdf-96ca-dd8d2d5e40d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:45.385506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ec464-f123-4bdf-96ca-dd8d2d5e40d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:45.385777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97ec464-f123-4bdf-96ca-dd8d2d5e40d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:45.395115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ec464-f123-4bdf-96ca-dd8d2d5e40d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:45.396568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97ec464-f123-4bdf-96ca-dd8d2d5e40d1 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:45.405985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1144/2298] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:19:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:19:47.991154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e4bc4-978f-43ee-a918-65c1a3598a84 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:48.007238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5e4bc4-978f-43ee-a918-65c1a3598a84 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:48.013497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad63c00-7fb5-4398-8c43-b522691871da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:48.014399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad63c00-7fb5-4398-8c43-b522691871da service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:48.014851 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad63c00-7fb5-4398-8c43-b522691871da service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:48.015335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad63c00-7fb5-4398-8c43-b522691871da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:48.017308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9ad63c00-7fb5-4398-8c43-b522691871da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:48.032396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad63c00-7fb5-4398-8c43-b522691871da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:48.034605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad63c00-7fb5-4398-8c43-b522691871da service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:48.052327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1155/2299] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:19:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:19:48.563390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ec464-f123-4bdf-96ca-dd8d2d5e40d1 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:19:48.585814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97ec464-f123-4bdf-96ca-dd8d2d5e40d1 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:19:48.600193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80b00b-ec54-402c-986d-e5fd3fdf30e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:19:48.603201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80b00b-ec54-402c-986d-e5fd3fdf30e5 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:19:48.604043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80b00b-ec54-402c-986d-e5fd3fdf30e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:19:48.617328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80b00b-ec54-402c-986d-e5fd3fdf30e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:19:48.627754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3d80b00b-ec54-402c-986d-e5fd3fdf30e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:19:48.659021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80b00b-ec54-402c-986d-e5fd3fdf30e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:19:48.660689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80b00b-ec54-402c-986d-e5fd3fdf30e5 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:19:48.685756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1145/2300] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:19:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:19:58.045975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=34 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:19:58.675065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=40 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:20:00.963578 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95138239-7c3d-4e51-ad56-ef5344972ce5 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:20:00.964079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95138239-7c3d-4e51-ad56-ef5344972ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:20:00.964211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95138239-7c3d-4e51-ad56-ef5344972ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:20:00.983396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-95138239-7c3d-4e51-ad56-ef5344972ce5 None None] MFA Rules not processed for user `db2f1171c5394a1c8879cd09c1e685b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:20:00.984614 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-95138239-7c3d-4e51-ad56-ef5344972ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:20:00.991700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1156/2301] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:20:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:20:01.298764 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80b00b-ec54-402c-986d-e5fd3fdf30e5 service nova] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:20:01.308430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80b00b-ec54-402c-986d-e5fd3fdf30e5 service nova] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:20:01.312025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cc4b3-a7e6-445f-82ce-b9ae6f034803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:20:01.312614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cc4b3-a7e6-445f-82ce-b9ae6f034803 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:20:01.312891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cc4b3-a7e6-445f-82ce-b9ae6f034803 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:20:01.313160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cc4b3-a7e6-445f-82ce-b9ae6f034803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:20:01.314825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f18cc4b3-a7e6-445f-82ce-b9ae6f034803 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:20:01.324018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cc4b3-a7e6-445f-82ce-b9ae6f034803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:20:01.325442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18cc4b3-a7e6-445f-82ce-b9ae6f034803 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:20:01.333199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1146/2302] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:20:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:20:03.242978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1789df-8719-4eaf-bfd0-3458b7f5eaea None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:20:03.243431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1789df-8719-4eaf-bfd0-3458b7f5eaea None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:20:03.243688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1789df-8719-4eaf-bfd0-3458b7f5eaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:20:03.263545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fd1789df-8719-4eaf-bfd0-3458b7f5eaea None None] MFA Rules not processed for user `8f96c9a2a49047b5b8abd3b2c1475a18`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:20:03.264742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fd1789df-8719-4eaf-bfd0-3458b7f5eaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:20:03.271655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1157/2303] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:20:03 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:20:11.329816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=6 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:20:13.257483 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=6 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:20:37.225067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb611b-f7f5-4bd0-a088-65b6e84c054f None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:20:37.225598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb611b-f7f5-4bd0-a088-65b6e84c054f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:20:37.225696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb611b-f7f5-4bd0-a088-65b6e84c054f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:20:37.244689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-36cb611b-f7f5-4bd0-a088-65b6e84c054f None None] MFA Rules not processed for user `810f0fa1f7774b37a09a91964c40b823`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:20:37.245874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-36cb611b-f7f5-4bd0-a088-65b6e84c054f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:20:37.253504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1147/2304] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:20:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:20:47.239869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:26.860671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df69fb8-2efe-4c8f-b1c1-a3bb36a81d7a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:26.861211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df69fb8-2efe-4c8f-b1c1-a3bb36a81d7a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:26.861211 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df69fb8-2efe-4c8f-b1c1-a3bb36a81d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:26.884443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-5df69fb8-2efe-4c8f-b1c1-a3bb36a81d7a None None] MFA Rules not processed for user `8ea8ba61dfa54cbc9a45543d20f31e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:26.886261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5df69fb8-2efe-4c8f-b1c1-a3bb36a81d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:26.894213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1158/2305] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:36.877907 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:37.340545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb611b-f7f5-4bd0-a088-65b6e84c054f None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:37.360423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb611b-f7f5-4bd0-a088-65b6e84c054f None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:37.364379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbeec1-fcf2-4a30-993a-38678c08e6e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:37.364983 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbeec1-fcf2-4a30-993a-38678c08e6e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:37.365264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbeec1-fcf2-4a30-993a-38678c08e6e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:37.365534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbeec1-fcf2-4a30-993a-38678c08e6e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:37.367272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e2fbeec1-fcf2-4a30-993a-38678c08e6e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:37.491187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbeec1-fcf2-4a30-993a-38678c08e6e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:37.492657 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbeec1-fcf2-4a30-993a-38678c08e6e3 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:37.512665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1148/2306] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:21:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:21:40.698411 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc5f4b-f9d7-491a-998a-ecd5b5977493 None None] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:40.698989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc5f4b-f9d7-491a-998a-ecd5b5977493 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:40.699168 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdc5f4b-f9d7-491a-998a-ecd5b5977493 None None] PATH_INFO: `/` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:40.700826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1159/2307] 10.222.0.26 () {54 vars in 961 bytes} [Sat Sep 10 14:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 10 14:21:40.706678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00c5fc-8054-4314-a36e-de3752646f91 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:40.707207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00c5fc-8054-4314-a36e-de3752646f91 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:40.707321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e00c5fc-8054-4314-a36e-de3752646f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:40.797003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-1e00c5fc-8054-4314-a36e-de3752646f91 None None] MFA Rules not processed for user `2a6a70a10cfa49348cc8b2b42870909d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:40.799617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1e00c5fc-8054-4314-a36e-de3752646f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:40.839044 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1149/2308] 10.222.0.26 () {58 vars in 1058 bytes} [Sat Sep 10 14:21:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:40.851588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc5f4b-f9d7-491a-998a-ecd5b5977493 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:40.873058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdc5f4b-f9d7-491a-998a-ecd5b5977493 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:40.880728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a06cd0-8da1-4d72-9b45-72d7b76f1b3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:40.881651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a06cd0-8da1-4d72-9b45-72d7b76f1b3e service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:40.882597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a06cd0-8da1-4d72-9b45-72d7b76f1b3e service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:40.883039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a06cd0-8da1-4d72-9b45-72d7b76f1b3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:40.886228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-65a06cd0-8da1-4d72-9b45-72d7b76f1b3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:40.901671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a06cd0-8da1-4d72-9b45-72d7b76f1b3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:40.903870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a06cd0-8da1-4d72-9b45-72d7b76f1b3e service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:40.916902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1160/2309] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:21:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:21:41.037521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00c5fc-8054-4314-a36e-de3752646f91 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:41.039025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-1e00c5fc-8054-4314-a36e-de3752646f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:41.054929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e00c5fc-8054-4314-a36e-de3752646f91 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:41.060050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:41.060705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:41.060968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:41.061237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] PATH_INFO: `/v3/users/8f96c9a2a49047b5b8abd3b2c1475a18` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:41.062731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f96c9a2a49047b5b8abd3b2c1475a18)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:41.065311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:41.080532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:41.116613 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f96c9a2a49047b5b8abd3b2c1475a18'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:41.117259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f96c9a2a49047b5b8abd3b2c1475a18'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:41.124738 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f96c9a2a49047b5b8abd3b2c1475a18'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:41.132926 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f96c9a2a49047b5b8abd3b2c1475a18'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:41.133572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-906b6602-4f47-4fc5-a126-be91fead3d09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f96c9a2a49047b5b8abd3b2c1475a18'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:41.150561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1150/2310] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:21:41 2022] DELETE /identity/v3/users/8f96c9a2a49047b5b8abd3b2c1475a18 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:21:41.325796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a06cd0-8da1-4d72-9b45-72d7b76f1b3e service placement] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:41.336901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a06cd0-8da1-4d72-9b45-72d7b76f1b3e service placement] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:41.411049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:41.411778 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:41.412058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:41.412334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] PATH_INFO: `/v3/projects/cffd972dec364c89bbc04fd078dc7fd9` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:41.415985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cffd972dec364c89bbc04fd078dc7fd9)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:41.417965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:41.480640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cffd972dec364c89bbc04fd078dc7fd9', 'request_id': 'req-184ef9ce-de64-4e91-a10a-d3393ca85948'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:41.486515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cffd972dec364c89bbc04fd078dc7fd9', 'request_id': 'req-184ef9ce-de64-4e91-a10a-d3393ca85948'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:41.494976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cffd972dec364c89bbc04fd078dc7fd9', 'request_id': 'req-184ef9ce-de64-4e91-a10a-d3393ca85948'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:41.508966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] The token cache is being invalidate because project cffd972dec364c89bbc04fd078dc7fd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:21:41.509396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:41.510262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-184ef9ce-de64-4e91-a10a-d3393ca85948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:41.512082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1161/2311] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:21:41 2022] DELETE /identity/v3/projects/cffd972dec364c89bbc04fd078dc7fd9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:21:41.517737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e51ca3-fa3e-4290-ae20-ef7c3ed8fcfb None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:41.518031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e51ca3-fa3e-4290-ae20-ef7c3ed8fcfb None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:41.518310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e51ca3-fa3e-4290-ae20-ef7c3ed8fcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:41.571082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d4e51ca3-fa3e-4290-ae20-ef7c3ed8fcfb None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:41.572350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d4e51ca3-fa3e-4290-ae20-ef7c3ed8fcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:41.766936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1151/2312] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:41.775194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c301b7-96c6-4416-90b2-eb37baa13e7f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:41.775681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c301b7-96c6-4416-90b2-eb37baa13e7f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:41.775781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c301b7-96c6-4416-90b2-eb37baa13e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:41.797145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-47c301b7-96c6-4416-90b2-eb37baa13e7f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:41.799402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-47c301b7-96c6-4416-90b2-eb37baa13e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:41.807979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1162/2313] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:41.812252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e51ca3-fa3e-4290-ae20-ef7c3ed8fcfb None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:41.831386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e51ca3-fa3e-4290-ae20-ef7c3ed8fcfb None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:41.843572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e3c43-c12b-49d0-8593-e6e324f9123e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:41.843572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e3c43-c12b-49d0-8593-e6e324f9123e admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:41.843572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e3c43-c12b-49d0-8593-e6e324f9123e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:41.843572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8e3c43-c12b-49d0-8593-e6e324f9123e admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:41.844043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e3c43-c12b-49d0-8593-e6e324f9123e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:21:41.844043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e3c43-c12b-49d0-8593-e6e324f9123e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:41.846222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8e3c43-c12b-49d0-8593-e6e324f9123e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:41.859769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1152/2314] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:21:41.864900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c301b7-96c6-4416-90b2-eb37baa13e7f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:41.875054 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c301b7-96c6-4416-90b2-eb37baa13e7f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:41.880044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e957c-8bab-4e5e-a1c2-833259b5e545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:41.880860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e957c-8bab-4e5e-a1c2-833259b5e545 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:41.881539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e957c-8bab-4e5e-a1c2-833259b5e545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:41.882024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26e957c-8bab-4e5e-a1c2-833259b5e545 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:41.882814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e957c-8bab-4e5e-a1c2-833259b5e545 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:41.885333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26e957c-8bab-4e5e-a1c2-833259b5e545 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:41.922955 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1163/2315] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:21:41 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:21:41.926937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e3c43-c12b-49d0-8593-e6e324f9123e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:41.937456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8e3c43-c12b-49d0-8593-e6e324f9123e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:41.995430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:42.003995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9f067-f7b3-4d32-b04d-0775fc065480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:42.004746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9f067-f7b3-4d32-b04d-0775fc065480 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:42.005211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9f067-f7b3-4d32-b04d-0775fc065480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:42.005659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac9f067-f7b3-4d32-b04d-0775fc065480 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:42.006800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9f067-f7b3-4d32-b04d-0775fc065480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:42.009168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac9f067-f7b3-4d32-b04d-0775fc065480 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:42.033247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1153/2316] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:21:41 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:21:42.037318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e957c-8bab-4e5e-a1c2-833259b5e545 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:42.050969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26e957c-8bab-4e5e-a1c2-833259b5e545 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:42.056200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c07ca-0413-486d-81c9-2f1b3573fe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:42.056882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c07ca-0413-486d-81c9-2f1b3573fe4d admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:42.057231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c07ca-0413-486d-81c9-2f1b3573fe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:42.057567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c07ca-0413-486d-81c9-2f1b3573fe4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:42.059435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c07ca-0413-486d-81c9-2f1b3573fe4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:42.061080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c07ca-0413-486d-81c9-2f1b3573fe4d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:42.073990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1164/2317] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:21:42.079358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9f067-f7b3-4d32-b04d-0775fc065480 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:42.090218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac9f067-f7b3-4d32-b04d-0775fc065480 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:42.095872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cabacf-5b2c-4607-a418-44a2275a74ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:42.096629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cabacf-5b2c-4607-a418-44a2275a74ba admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:42.097497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cabacf-5b2c-4607-a418-44a2275a74ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:42.097497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cabacf-5b2c-4607-a418-44a2275a74ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:42.098179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cabacf-5b2c-4607-a418-44a2275a74ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:42.099940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cabacf-5b2c-4607-a418-44a2275a74ba admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:42.110860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1154/2318] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:21:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:21:42.115411 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c07ca-0413-486d-81c9-2f1b3573fe4d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:42.127872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c07ca-0413-486d-81c9-2f1b3573fe4d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:42.133976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9709f7-2bee-4601-a075-6e2ec712c7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:42.134643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9709f7-2bee-4601-a075-6e2ec712c7ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:42.134926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9709f7-2bee-4601-a075-6e2ec712c7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:42.135199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9709f7-2bee-4601-a075-6e2ec712c7ef admin admin] PATH_INFO: `/v3/projects/5ba6e747296f47119282c836ef7ddc4e/users/fd369300973c4c6d9f570c323edcd23f/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:42.151320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9709f7-2bee-4601-a075-6e2ec712c7ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba6e747296f47119282c836ef7ddc4e, user_id=fd369300973c4c6d9f570c323edcd23f, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:42.154036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9709f7-2bee-4601-a075-6e2ec712c7ef admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:42.171156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1165/2319] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:21:42 2022] PUT /identity/v3/projects/5ba6e747296f47119282c836ef7ddc4e/users/fd369300973c4c6d9f570c323edcd23f/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:21:42.182018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49be357-2a1b-40fe-b09d-fb5945e58eee None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:42.182399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49be357-2a1b-40fe-b09d-fb5945e58eee None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:42.182758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49be357-2a1b-40fe-b09d-fb5945e58eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:42.212124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c49be357-2a1b-40fe-b09d-fb5945e58eee None None] MFA Rules not processed for user `fd369300973c4c6d9f570c323edcd23f`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:42.213373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c49be357-2a1b-40fe-b09d-fb5945e58eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:42.278919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1155/2320] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:42.294392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9709f7-2bee-4601-a075-6e2ec712c7ef admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:42.298035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5a9709f7-2bee-4601-a075-6e2ec712c7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:42.317069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:42.406248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9709f7-2bee-4601-a075-6e2ec712c7ef admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:42.412007 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec56b2a-0cbd-4294-b909-c395be962d61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:42.412686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56b2a-0cbd-4294-b909-c395be962d61 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:42.412883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56b2a-0cbd-4294-b909-c395be962d61 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:42.413153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec56b2a-0cbd-4294-b909-c395be962d61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:42.421651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec56b2a-0cbd-4294-b909-c395be962d61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:42.423591 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec56b2a-0cbd-4294-b909-c395be962d61 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:42.433531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1166/2321] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:21:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:21:48.705527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49be357-2a1b-40fe-b09d-fb5945e58eee None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:48.706906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c49be357-2a1b-40fe-b09d-fb5945e58eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:48.779012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49be357-2a1b-40fe-b09d-fb5945e58eee None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:48.784931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65626718-5e46-4b97-b6b7-ba82f269ac90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:48.785533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65626718-5e46-4b97-b6b7-ba82f269ac90 service placement] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:48.785812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65626718-5e46-4b97-b6b7-ba82f269ac90 service placement] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:48.786082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65626718-5e46-4b97-b6b7-ba82f269ac90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:48.787870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-65626718-5e46-4b97-b6b7-ba82f269ac90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:48.799124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65626718-5e46-4b97-b6b7-ba82f269ac90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:48.800749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65626718-5e46-4b97-b6b7-ba82f269ac90 service placement] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:48.810433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1156/2322] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:21:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:21:51.862264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec56b2a-0cbd-4294-b909-c395be962d61 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:51.924746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec56b2a-0cbd-4294-b909-c395be962d61 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:51.929301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:51.929932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:51.930208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:51.930481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] PATH_INFO: `/v3/users/fd369300973c4c6d9f570c323edcd23f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:51.931636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd369300973c4c6d9f570c323edcd23f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:51.933584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:51.979488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd369300973c4c6d9f570c323edcd23f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:51.985821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd369300973c4c6d9f570c323edcd23f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:51.986413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd369300973c4c6d9f570c323edcd23f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:51.987051 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd369300973c4c6d9f570c323edcd23f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:51.994363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd369300973c4c6d9f570c323edcd23f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:52.002798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:52.014133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1167/2323] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:21:51 2022] DELETE /identity/v3/users/fd369300973c4c6d9f570c323edcd23f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:21:52.019381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0cc04-59aa-4645-a7ea-54256208d3ed None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:52.019885 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0cc04-59aa-4645-a7ea-54256208d3ed None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:52.019997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e0cc04-59aa-4645-a7ea-54256208d3ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:52.038270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d7e0cc04-59aa-4645-a7ea-54256208d3ed None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:52.039492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d7e0cc04-59aa-4645-a7ea-54256208d3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:52.086208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:52.098403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1157/2324] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:52.108190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:52.109585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:52.167664 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab22eb1f-7f8f-4001-941f-80c8a831f9a3 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:52.171277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee57e80-ea49-4c32-a534-220200c4997d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:52.171906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee57e80-ea49-4c32-a534-220200c4997d service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:52.172187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee57e80-ea49-4c32-a534-220200c4997d service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:52.172453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee57e80-ea49-4c32-a534-220200c4997d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:52.181416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee57e80-ea49-4c32-a534-220200c4997d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:52.182847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee57e80-ea49-4c32-a534-220200c4997d service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:52.193122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1168/2325] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:21:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:21:52.465400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0cc04-59aa-4645-a7ea-54256208d3ed None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:52.466851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d7e0cc04-59aa-4645-a7ea-54256208d3ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:52.478975 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e0cc04-59aa-4645-a7ea-54256208d3ed None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:52.483584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:52.484215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:52.484491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:52.485295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] PATH_INFO: `/v3/projects/5ba6e747296f47119282c836ef7ddc4e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:52.486436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba6e747296f47119282c836ef7ddc4e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:52.488664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:52.540013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba6e747296f47119282c836ef7ddc4e', 'request_id': 'req-b745f4a0-69b9-4823-a065-f1a2220a91e1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:52.548179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba6e747296f47119282c836ef7ddc4e', 'request_id': 'req-b745f4a0-69b9-4823-a065-f1a2220a91e1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:52.554314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba6e747296f47119282c836ef7ddc4e', 'request_id': 'req-b745f4a0-69b9-4823-a065-f1a2220a91e1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:52.568594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] The token cache is being invalidate because project 5ba6e747296f47119282c836ef7ddc4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:21:52.569069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:52.570021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b745f4a0-69b9-4823-a065-f1a2220a91e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:52.572499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1158/2326] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:21:52 2022] DELETE /identity/v3/projects/5ba6e747296f47119282c836ef7ddc4e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:21:52.578497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d627-44a3-4631-a7f4-72d326e24379 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:52.578938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d627-44a3-4631-a7f4-72d326e24379 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:52.579214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229d627-44a3-4631-a7f4-72d326e24379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:52.604476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3229d627-44a3-4631-a7f4-72d326e24379 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:52.605686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3229d627-44a3-4631-a7f4-72d326e24379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:52.675995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1169/2327] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:52.685184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a812-e177-4fb3-8291-c4110825bbb9 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:52.685468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a812-e177-4fb3-8291-c4110825bbb9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:52.685744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9a812-e177-4fb3-8291-c4110825bbb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:52.711324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-b4c9a812-e177-4fb3-8291-c4110825bbb9 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:52.712658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b4c9a812-e177-4fb3-8291-c4110825bbb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:52.721456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1159/2328] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:52.725778 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229d627-44a3-4631-a7f4-72d326e24379 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:52.738842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229d627-44a3-4631-a7f4-72d326e24379 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:52.744397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202377a9-b753-4dbb-ad05-33125069c16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:52.744998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202377a9-b753-4dbb-ad05-33125069c16f admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:52.745277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202377a9-b753-4dbb-ad05-33125069c16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:52.745565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202377a9-b753-4dbb-ad05-33125069c16f admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:52.746220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202377a9-b753-4dbb-ad05-33125069c16f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:21:52.746523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202377a9-b753-4dbb-ad05-33125069c16f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:52.749369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202377a9-b753-4dbb-ad05-33125069c16f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:52.767153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1170/2329] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:21:52.771633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a812-e177-4fb3-8291-c4110825bbb9 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:52.784847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9a812-e177-4fb3-8291-c4110825bbb9 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:52.789688 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427bd00-d80d-4cf5-a794-25a60c7ca749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:52.790334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427bd00-d80d-4cf5-a794-25a60c7ca749 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:52.790557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427bd00-d80d-4cf5-a794-25a60c7ca749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:52.790827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d427bd00-d80d-4cf5-a794-25a60c7ca749 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:52.791497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427bd00-d80d-4cf5-a794-25a60c7ca749 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:52.793390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d427bd00-d80d-4cf5-a794-25a60c7ca749 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:52.826319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1160/2330] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:21:52 2022] POST /identity/v3/projects => generated 401 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:21:52.830744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202377a9-b753-4dbb-ad05-33125069c16f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:52.841346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202377a9-b753-4dbb-ad05-33125069c16f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:52.901695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa32ab7-2d9e-4066-a921-42bf263bc59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:52.902371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa32ab7-2d9e-4066-a921-42bf263bc59c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:52.902719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa32ab7-2d9e-4066-a921-42bf263bc59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:52.902917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa32ab7-2d9e-4066-a921-42bf263bc59c admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:52.903621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa32ab7-2d9e-4066-a921-42bf263bc59c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:52.905558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa32ab7-2d9e-4066-a921-42bf263bc59c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:52.930263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1171/2331] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:21:52 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:21:52.934573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427bd00-d80d-4cf5-a794-25a60c7ca749 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:52.945033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d427bd00-d80d-4cf5-a794-25a60c7ca749 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:52.949823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105dfd3-3c3f-40b6-8c0d-141cb98cea1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:52.950421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105dfd3-3c3f-40b6-8c0d-141cb98cea1a admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:52.950697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105dfd3-3c3f-40b6-8c0d-141cb98cea1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:52.950970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0105dfd3-3c3f-40b6-8c0d-141cb98cea1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:52.951648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105dfd3-3c3f-40b6-8c0d-141cb98cea1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:52.953260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0105dfd3-3c3f-40b6-8c0d-141cb98cea1a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:52.963567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1161/2332] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:21:52.967643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa32ab7-2d9e-4066-a921-42bf263bc59c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:52.978457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa32ab7-2d9e-4066-a921-42bf263bc59c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:52.983031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67038f-1450-4a0a-a66c-1b1a9ebff38c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:52.983673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67038f-1450-4a0a-a66c-1b1a9ebff38c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:52.983940 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67038f-1450-4a0a-a66c-1b1a9ebff38c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:52.984217 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67038f-1450-4a0a-a66c-1b1a9ebff38c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:52.985522 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67038f-1450-4a0a-a66c-1b1a9ebff38c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:52.987520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67038f-1450-4a0a-a66c-1b1a9ebff38c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:52.993361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:53.002419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1172/2333] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:21:53.003804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105dfd3-3c3f-40b6-8c0d-141cb98cea1a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:53.014149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0105dfd3-3c3f-40b6-8c0d-141cb98cea1a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:53.020016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5a0a7-044b-4a90-94df-c8b501bd4b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:53.020960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5a0a7-044b-4a90-94df-c8b501bd4b87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.020960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5a0a7-044b-4a90-94df-c8b501bd4b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.021306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d5a0a7-044b-4a90-94df-c8b501bd4b87 admin admin] PATH_INFO: `/v3/projects/4d18999b6dcd4953b1d20f25f22bbc1c/users/7e8b0ae2bb8e4709b985a4f08a5ef504/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.036748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5a0a7-044b-4a90-94df-c8b501bd4b87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d18999b6dcd4953b1d20f25f22bbc1c, user_id=7e8b0ae2bb8e4709b985a4f08a5ef504, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:53.039652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d5a0a7-044b-4a90-94df-c8b501bd4b87 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:53.053237 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1162/2334] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:21:53 2022] PUT /identity/v3/projects/4d18999b6dcd4953b1d20f25f22bbc1c/users/7e8b0ae2bb8e4709b985a4f08a5ef504/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:21:53.065310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb062a1-23bf-4acd-93b4-f28ff8ae5d7f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.065596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb062a1-23bf-4acd-93b4-f28ff8ae5d7f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.065874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb062a1-23bf-4acd-93b4-f28ff8ae5d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.095928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8cb062a1-23bf-4acd-93b4-f28ff8ae5d7f None None] MFA Rules not processed for user `7e8b0ae2bb8e4709b985a4f08a5ef504`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:53.097156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8cb062a1-23bf-4acd-93b4-f28ff8ae5d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:53.165346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1173/2335] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:53 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:53.170087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5a0a7-044b-4a90-94df-c8b501bd4b87 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:53.239697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d5a0a7-044b-4a90-94df-c8b501bd4b87 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:53.245078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b37cf-b88c-4896-8f29-82bb7448c273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:53.245673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b37cf-b88c-4896-8f29-82bb7448c273 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.245943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b37cf-b88c-4896-8f29-82bb7448c273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.246208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b37cf-b88c-4896-8f29-82bb7448c273 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.246951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b37cf-b88c-4896-8f29-82bb7448c273 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:53.248771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b37cf-b88c-4896-8f29-82bb7448c273 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:53.276445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:53.285250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1163/2336] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:21:53 2022] POST /identity/v3/projects => generated 401 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:21:53.290299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb062a1-23bf-4acd-93b4-f28ff8ae5d7f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:53.302052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb062a1-23bf-4acd-93b4-f28ff8ae5d7f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:53.365448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bcfd9-b76b-4180-a0f1-c7b21f6b3f2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:53.366216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bcfd9-b76b-4180-a0f1-c7b21f6b3f2f admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.366492 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bcfd9-b76b-4180-a0f1-c7b21f6b3f2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.366760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04bcfd9-b76b-4180-a0f1-c7b21f6b3f2f admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.367393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bcfd9-b76b-4180-a0f1-c7b21f6b3f2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:53.369335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04bcfd9-b76b-4180-a0f1-c7b21f6b3f2f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:53.395784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1174/2337] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:21:53 2022] POST /identity/v3/users => generated 307 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:21:53.400016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b37cf-b88c-4896-8f29-82bb7448c273 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:53.411136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b37cf-b88c-4896-8f29-82bb7448c273 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:53.415767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca982be2-068c-4772-aa8d-28293a1a9786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:53.416361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca982be2-068c-4772-aa8d-28293a1a9786 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.416635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca982be2-068c-4772-aa8d-28293a1a9786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.416903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca982be2-068c-4772-aa8d-28293a1a9786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.417631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca982be2-068c-4772-aa8d-28293a1a9786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:53.419224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca982be2-068c-4772-aa8d-28293a1a9786 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:53.430658 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1164/2338] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:21:53.435031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bcfd9-b76b-4180-a0f1-c7b21f6b3f2f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:53.445534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04bcfd9-b76b-4180-a0f1-c7b21f6b3f2f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:53.450112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6ca7f-7f5a-43a0-831c-bcb0b25f6060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:53.450763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6ca7f-7f5a-43a0-831c-bcb0b25f6060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.451038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6ca7f-7f5a-43a0-831c-bcb0b25f6060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.451312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6ca7f-7f5a-43a0-831c-bcb0b25f6060 admin admin] PATH_INFO: `/v3/projects/cbcbe6a07a254af9855528a13d64f4d5/users/fec29746855f456a9c9496f4ba5c7968/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.466509 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6ca7f-7f5a-43a0-831c-bcb0b25f6060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbcbe6a07a254af9855528a13d64f4d5, user_id=fec29746855f456a9c9496f4ba5c7968, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:53.469795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6ca7f-7f5a-43a0-831c-bcb0b25f6060 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:53.483547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1175/2339] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:21:53 2022] PUT /identity/v3/projects/cbcbe6a07a254af9855528a13d64f4d5/users/fec29746855f456a9c9496f4ba5c7968/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:21:53.488660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca982be2-068c-4772-aa8d-28293a1a9786 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:53.499253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca982be2-068c-4772-aa8d-28293a1a9786 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:53.555967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf479607-6dd9-4ab0-b945-fcd79299ca34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:53.556573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf479607-6dd9-4ab0-b945-fcd79299ca34 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.556848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf479607-6dd9-4ab0-b945-fcd79299ca34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.557118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf479607-6dd9-4ab0-b945-fcd79299ca34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.557803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf479607-6dd9-4ab0-b945-fcd79299ca34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:53.559401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf479607-6dd9-4ab0-b945-fcd79299ca34 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:53.569435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1165/2340] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:21:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:21:53.573739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6ca7f-7f5a-43a0-831c-bcb0b25f6060 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:53.584130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6ca7f-7f5a-43a0-831c-bcb0b25f6060 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:53.588767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79dadb7-7d9a-47e2-9171-cb0e12f06358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:53.589409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dadb7-7d9a-47e2-9171-cb0e12f06358 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.589686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dadb7-7d9a-47e2-9171-cb0e12f06358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.589957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79dadb7-7d9a-47e2-9171-cb0e12f06358 admin admin] PATH_INFO: `/v3/projects/cbcbe6a07a254af9855528a13d64f4d5/users/fec29746855f456a9c9496f4ba5c7968/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.591939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79dadb7-7d9a-47e2-9171-cb0e12f06358 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbcbe6a07a254af9855528a13d64f4d5, user_id=fec29746855f456a9c9496f4ba5c7968, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:53.595311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79dadb7-7d9a-47e2-9171-cb0e12f06358 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:53.605405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1176/2341] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:21:53 2022] PUT /identity/v3/projects/cbcbe6a07a254af9855528a13d64f4d5/users/fec29746855f456a9c9496f4ba5c7968/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:21:53.622665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ce324-0097-46dd-ac59-e05ec00e8b74 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.622855 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ce324-0097-46dd-ac59-e05ec00e8b74 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.623127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ce324-0097-46dd-ac59-e05ec00e8b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.651502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-fd5ce324-0097-46dd-ac59-e05ec00e8b74 None None] MFA Rules not processed for user `fec29746855f456a9c9496f4ba5c7968`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:53.652716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-fd5ce324-0097-46dd-ac59-e05ec00e8b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:53.722428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1166/2342] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:53 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:53.732480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79dadb7-7d9a-47e2-9171-cb0e12f06358 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:53.735248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e79dadb7-7d9a-47e2-9171-cb0e12f06358 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:53.781183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:53.795049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79dadb7-7d9a-47e2-9171-cb0e12f06358 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:53.798670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71a50-e5d3-43a1-a0f9-84314b31cc87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:53.799255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71a50-e5d3-43a1-a0f9-84314b31cc87 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.799560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71a50-e5d3-43a1-a0f9-84314b31cc87 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.799834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f71a50-e5d3-43a1-a0f9-84314b31cc87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.808499 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f71a50-e5d3-43a1-a0f9-84314b31cc87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:53.809937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f71a50-e5d3-43a1-a0f9-84314b31cc87 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:53.819179 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1177/2343] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:21:53 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:21:53.896650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ce324-0097-46dd-ac59-e05ec00e8b74 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:53.907376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ce324-0097-46dd-ac59-e05ec00e8b74 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:53.911188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4040f7f6-d617-470b-ab6e-f57bff8ee149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:53.911845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040f7f6-d617-470b-ab6e-f57bff8ee149 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:53.912121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040f7f6-d617-470b-ab6e-f57bff8ee149 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:53.912388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4040f7f6-d617-470b-ab6e-f57bff8ee149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:53.979706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040f7f6-d617-470b-ab6e-f57bff8ee149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:53.981131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4040f7f6-d617-470b-ab6e-f57bff8ee149 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:53.990227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1167/2344] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:21:53 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:21:58.073317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71a50-e5d3-43a1-a0f9-84314b31cc87 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:58.139544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f71a50-e5d3-43a1-a0f9-84314b31cc87 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:58.144652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:58.145266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:58.145537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:58.145804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] PATH_INFO: `/v3/users/7e8b0ae2bb8e4709b985a4f08a5ef504` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:58.146897 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e8b0ae2bb8e4709b985a4f08a5ef504)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:58.148841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:58.196997 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e8b0ae2bb8e4709b985a4f08a5ef504'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.203798 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e8b0ae2bb8e4709b985a4f08a5ef504'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.204445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e8b0ae2bb8e4709b985a4f08a5ef504'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.205054 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e8b0ae2bb8e4709b985a4f08a5ef504'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.212224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e8b0ae2bb8e4709b985a4f08a5ef504'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.229587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1178/2345] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:21:58 2022] DELETE /identity/v3/users/7e8b0ae2bb8e4709b985a4f08a5ef504 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:21:58.234125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b2719-2ba8-460b-a028-aaed3bf7b031 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:58.235155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b2719-2ba8-460b-a028-aaed3bf7b031 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:58.235335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6b2719-2ba8-460b-a028-aaed3bf7b031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:58.256026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-9f6b2719-2ba8-460b-a028-aaed3bf7b031 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:58.257389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9f6b2719-2ba8-460b-a028-aaed3bf7b031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:58.262786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:58.321282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1168/2346] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:58.332122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:58.333545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:58.408668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:58.418311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b0f6c-92c9-4f29-87e3-df18bbe5843a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:58.422479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076b1e3-c15d-4347-bc35-a8a9de67106c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:58.423087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076b1e3-c15d-4347-bc35-a8a9de67106c service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:58.423364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076b1e3-c15d-4347-bc35-a8a9de67106c service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:58.423668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076b1e3-c15d-4347-bc35-a8a9de67106c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:58.433598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076b1e3-c15d-4347-bc35-a8a9de67106c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:58.435046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2076b1e3-c15d-4347-bc35-a8a9de67106c service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:58.445201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1179/2347] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:21:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:21:58.717541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b2719-2ba8-460b-a028-aaed3bf7b031 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:58.720358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9f6b2719-2ba8-460b-a028-aaed3bf7b031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:58.755381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6b2719-2ba8-460b-a028-aaed3bf7b031 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:58.762592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:58.763485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:58.764015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:58.764452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] PATH_INFO: `/v3/projects/4d18999b6dcd4953b1d20f25f22bbc1c` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:58.766142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d18999b6dcd4953b1d20f25f22bbc1c)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:58.768597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:58.817957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d18999b6dcd4953b1d20f25f22bbc1c', 'request_id': 'req-643ae400-51af-4dd0-9156-04c3b64dd570'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.825691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d18999b6dcd4953b1d20f25f22bbc1c', 'request_id': 'req-643ae400-51af-4dd0-9156-04c3b64dd570'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.831886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d18999b6dcd4953b1d20f25f22bbc1c', 'request_id': 'req-643ae400-51af-4dd0-9156-04c3b64dd570'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.843136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] The token cache is being invalidate because project 4d18999b6dcd4953b1d20f25f22bbc1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:21:58.843422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.844650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.846153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1169/2348] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:21:58 2022] DELETE /identity/v3/projects/4d18999b6dcd4953b1d20f25f22bbc1c => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:21:58.850299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076b1e3-c15d-4347-bc35-a8a9de67106c service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:58.861297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2076b1e3-c15d-4347-bc35-a8a9de67106c service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:58.933930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:58.934653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:58.934926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:58.935193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] PATH_INFO: `/v3/users/fec29746855f456a9c9496f4ba5c7968` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:58.936393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fec29746855f456a9c9496f4ba5c7968)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:58.938904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:58.990831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fec29746855f456a9c9496f4ba5c7968'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.998519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fec29746855f456a9c9496f4ba5c7968'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:58.999712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fec29746855f456a9c9496f4ba5c7968'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:59.000562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fec29746855f456a9c9496f4ba5c7968'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:59.008096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c63acd7f-8a34-45db-8a01-8404d78fc1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fec29746855f456a9c9496f4ba5c7968'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:59.026581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1180/2349] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:21:58 2022] DELETE /identity/v3/users/fec29746855f456a9c9496f4ba5c7968 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:21:59.296186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:59.297805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:59.316402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ae400-51af-4dd0-9156-04c3b64dd570 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:59.374984 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:59.376298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:59.376590 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:59.376874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] PATH_INFO: `/v3/projects/cbcbe6a07a254af9855528a13d64f4d5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:59.377993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbcbe6a07a254af9855528a13d64f4d5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:59.379926 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:59.416430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:59.427996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbcbe6a07a254af9855528a13d64f4d5', 'request_id': 'req-af4401d4-1424-430b-ab1e-878af7f9feb9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:59.435381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbcbe6a07a254af9855528a13d64f4d5', 'request_id': 'req-af4401d4-1424-430b-ab1e-878af7f9feb9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:59.442058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbcbe6a07a254af9855528a13d64f4d5', 'request_id': 'req-af4401d4-1424-430b-ab1e-878af7f9feb9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:59.454350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] The token cache is being invalidate because project cbcbe6a07a254af9855528a13d64f4d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:21:59.454629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:59.455264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-af4401d4-1424-430b-ab1e-878af7f9feb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:21:59.456758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1170/2350] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:21:59 2022] DELETE /identity/v3/projects/cbcbe6a07a254af9855528a13d64f4d5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:21:59.461753 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b23bd-f166-4688-8071-2bf2e096e456 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:59.462201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b23bd-f166-4688-8071-2bf2e096e456 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:59.462318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b23bd-f166-4688-8071-2bf2e096e456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:59.481643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c87b23bd-f166-4688-8071-2bf2e096e456 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:59.482832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c87b23bd-f166-4688-8071-2bf2e096e456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:59.541945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1181/2351] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:59.549443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9810e-80de-493e-84fb-683bb5c7ffb5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:59.549718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9810e-80de-493e-84fb-683bb5c7ffb5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:59.549985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e9810e-80de-493e-84fb-683bb5c7ffb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:59.569819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a5e9810e-80de-493e-84fb-683bb5c7ffb5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:59.571023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a5e9810e-80de-493e-84fb-683bb5c7ffb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:59.579284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1171/2352] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:59.583535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b23bd-f166-4688-8071-2bf2e096e456 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:59.593110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b23bd-f166-4688-8071-2bf2e096e456 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:59.597923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1da1e6-5f89-4759-937b-658d04f6ce28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:59.598511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1da1e6-5f89-4759-937b-658d04f6ce28 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:59.598789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1da1e6-5f89-4759-937b-658d04f6ce28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:59.599065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1da1e6-5f89-4759-937b-658d04f6ce28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:59.599763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1da1e6-5f89-4759-937b-658d04f6ce28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:21:59.600062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1da1e6-5f89-4759-937b-658d04f6ce28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:59.601663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1da1e6-5f89-4759-937b-658d04f6ce28 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:59.603094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:21:59.615630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1182/2353] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:21:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:21:59.619541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9810e-80de-493e-84fb-683bb5c7ffb5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:59.629292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e9810e-80de-493e-84fb-683bb5c7ffb5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:59.633962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e669dc7-0fca-4e93-9fdd-5f8ca1a4fd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:59.634550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e669dc7-0fca-4e93-9fdd-5f8ca1a4fd7e admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:59.634825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e669dc7-0fca-4e93-9fdd-5f8ca1a4fd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:59.635091 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e669dc7-0fca-4e93-9fdd-5f8ca1a4fd7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:59.635766 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e669dc7-0fca-4e93-9fdd-5f8ca1a4fd7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:59.637652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e669dc7-0fca-4e93-9fdd-5f8ca1a4fd7e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:59.664610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1172/2354] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:21:59 2022] POST /identity/v3/projects => generated 389 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:21:59.668788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1da1e6-5f89-4759-937b-658d04f6ce28 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:59.678690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1da1e6-5f89-4759-937b-658d04f6ce28 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:59.734028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126358e5-2b93-4cf4-8926-03e45052c137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:59.735163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126358e5-2b93-4cf4-8926-03e45052c137 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:59.735434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126358e5-2b93-4cf4-8926-03e45052c137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:59.735727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126358e5-2b93-4cf4-8926-03e45052c137 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:59.736349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126358e5-2b93-4cf4-8926-03e45052c137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:59.738528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126358e5-2b93-4cf4-8926-03e45052c137 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:59.758998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1183/2355] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:21:59 2022] POST /identity/v3/users => generated 295 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:21:59.763106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e669dc7-0fca-4e93-9fdd-5f8ca1a4fd7e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:59.773696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e669dc7-0fca-4e93-9fdd-5f8ca1a4fd7e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:59.778235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff5ce0-a067-4e70-9c9e-cb4a441616eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:59.778841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff5ce0-a067-4e70-9c9e-cb4a441616eb admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:59.779112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff5ce0-a067-4e70-9c9e-cb4a441616eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:59.779380 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff5ce0-a067-4e70-9c9e-cb4a441616eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:59.780041 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff5ce0-a067-4e70-9c9e-cb4a441616eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:59.782273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff5ce0-a067-4e70-9c9e-cb4a441616eb admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:59.792298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1173/2356] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:21:59.796112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126358e5-2b93-4cf4-8926-03e45052c137 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:59.806003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126358e5-2b93-4cf4-8926-03e45052c137 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:59.810574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ea7cd-8ca9-4bec-b44a-7e7d0f4e1dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:59.811157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ea7cd-8ca9-4bec-b44a-7e7d0f4e1dbc admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:59.811461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ea7cd-8ca9-4bec-b44a-7e7d0f4e1dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:59.811739 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ea7cd-8ca9-4bec-b44a-7e7d0f4e1dbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:59.812367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ea7cd-8ca9-4bec-b44a-7e7d0f4e1dbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:59.813961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ea7cd-8ca9-4bec-b44a-7e7d0f4e1dbc admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:59.824236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1184/2357] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:21:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:21:59.828786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff5ce0-a067-4e70-9c9e-cb4a441616eb admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:21:59.838883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff5ce0-a067-4e70-9c9e-cb4a441616eb admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:21:59.843386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3a3b7-3387-4a02-a22f-951af1a758cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:21:59.844649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3a3b7-3387-4a02-a22f-951af1a758cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:59.845062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3a3b7-3387-4a02-a22f-951af1a758cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:59.845338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3a3b7-3387-4a02-a22f-951af1a758cc admin admin] PATH_INFO: `/v3/projects/a8ec220c0d1d496d8df58807675e8c7c/users/500ff8ecce8641d6add9602720accd69/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:59.859562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3a3b7-3387-4a02-a22f-951af1a758cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ec220c0d1d496d8df58807675e8c7c, user_id=500ff8ecce8641d6add9602720accd69, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:21:59.862378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3a3b7-3387-4a02-a22f-951af1a758cc admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:21:59.875428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1174/2358] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:21:59 2022] PUT /identity/v3/projects/a8ec220c0d1d496d8df58807675e8c7c/users/500ff8ecce8641d6add9602720accd69/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:21:59.886643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75984fcd-60a1-4cf9-add5-33531f42e6f1 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:21:59.886929 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75984fcd-60a1-4cf9-add5-33531f42e6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:21:59.887201 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75984fcd-60a1-4cf9-add5-33531f42e6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:21:59.916786 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-75984fcd-60a1-4cf9-add5-33531f42e6f1 None None] MFA Rules not processed for user `500ff8ecce8641d6add9602720accd69`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:21:59.918094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-75984fcd-60a1-4cf9-add5-33531f42e6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:21:59.981634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1185/2359] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:21:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:21:59.986440 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3a3b7-3387-4a02-a22f-951af1a758cc admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:00.047803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3a3b7-3387-4a02-a22f-951af1a758cc admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:00.052290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52aa6c-5936-4a8c-b3dc-b29e1b9d16bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:00.052907 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52aa6c-5936-4a8c-b3dc-b29e1b9d16bd admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:00.053180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52aa6c-5936-4a8c-b3dc-b29e1b9d16bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:00.053446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc52aa6c-5936-4a8c-b3dc-b29e1b9d16bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:00.054073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52aa6c-5936-4a8c-b3dc-b29e1b9d16bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:00.056069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc52aa6c-5936-4a8c-b3dc-b29e1b9d16bd admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:00.083195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1175/2360] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:21:59 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:22:00.087301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75984fcd-60a1-4cf9-add5-33531f42e6f1 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:00.098858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75984fcd-60a1-4cf9-add5-33531f42e6f1 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:00.155322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6119134-7f8e-469f-b70e-00e993d4948b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:00.155963 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6119134-7f8e-469f-b70e-00e993d4948b admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:00.156240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6119134-7f8e-469f-b70e-00e993d4948b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:00.156595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6119134-7f8e-469f-b70e-00e993d4948b admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:00.157237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6119134-7f8e-469f-b70e-00e993d4948b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:00.159177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6119134-7f8e-469f-b70e-00e993d4948b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:00.168398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:00.181872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1186/2361] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:22:00 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:22:00.185825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52aa6c-5936-4a8c-b3dc-b29e1b9d16bd admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:00.197103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc52aa6c-5936-4a8c-b3dc-b29e1b9d16bd admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:00.202202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099398a4-d827-44fc-bf35-fb8380d1ec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:00.202990 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099398a4-d827-44fc-bf35-fb8380d1ec73 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:00.204027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099398a4-d827-44fc-bf35-fb8380d1ec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:00.204478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099398a4-d827-44fc-bf35-fb8380d1ec73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:00.205504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099398a4-d827-44fc-bf35-fb8380d1ec73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:00.207103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099398a4-d827-44fc-bf35-fb8380d1ec73 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:00.219988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1176/2362] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:22:00.224170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6119134-7f8e-469f-b70e-00e993d4948b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:00.234348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6119134-7f8e-469f-b70e-00e993d4948b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:00.239569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fac37-cf82-4c39-9c8e-1139a80947a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:00.240213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fac37-cf82-4c39-9c8e-1139a80947a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:00.240488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fac37-cf82-4c39-9c8e-1139a80947a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:00.240768 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fac37-cf82-4c39-9c8e-1139a80947a0 admin admin] PATH_INFO: `/v3/projects/4fc9c98f39c747d8a60e35cc6c4e8039/users/bef537efe1d144ec96aee78d7acc851c/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:00.256023 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fac37-cf82-4c39-9c8e-1139a80947a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc9c98f39c747d8a60e35cc6c4e8039, user_id=bef537efe1d144ec96aee78d7acc851c, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:00.258708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fac37-cf82-4c39-9c8e-1139a80947a0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:00.271936 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1187/2363] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:22:00 2022] PUT /identity/v3/projects/4fc9c98f39c747d8a60e35cc6c4e8039/users/bef537efe1d144ec96aee78d7acc851c/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:00.275914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099398a4-d827-44fc-bf35-fb8380d1ec73 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:00.285734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099398a4-d827-44fc-bf35-fb8380d1ec73 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:00.315174 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:00.342713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1244b-71f2-4d95-b05c-421f5cdbacac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:00.343314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1244b-71f2-4d95-b05c-421f5cdbacac admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:00.343685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1244b-71f2-4d95-b05c-421f5cdbacac admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:00.344122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1244b-71f2-4d95-b05c-421f5cdbacac admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:00.344978 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1244b-71f2-4d95-b05c-421f5cdbacac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:00.346574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1244b-71f2-4d95-b05c-421f5cdbacac admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:00.356022 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1177/2364] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:22:00.360393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fac37-cf82-4c39-9c8e-1139a80947a0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:00.370801 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fac37-cf82-4c39-9c8e-1139a80947a0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:00.375852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb4228-0f96-43a0-bfcb-55425051d4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:00.376487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb4228-0f96-43a0-bfcb-55425051d4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:00.376762 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb4228-0f96-43a0-bfcb-55425051d4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:00.377030 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cb4228-0f96-43a0-bfcb-55425051d4d4 admin admin] PATH_INFO: `/v3/projects/4fc9c98f39c747d8a60e35cc6c4e8039/users/bef537efe1d144ec96aee78d7acc851c/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:00.378899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb4228-0f96-43a0-bfcb-55425051d4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc9c98f39c747d8a60e35cc6c4e8039, user_id=bef537efe1d144ec96aee78d7acc851c, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:00.381582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cb4228-0f96-43a0-bfcb-55425051d4d4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:00.392060 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1188/2365] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:22:00 2022] PUT /identity/v3/projects/4fc9c98f39c747d8a60e35cc6c4e8039/users/bef537efe1d144ec96aee78d7acc851c/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:00.402224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35ccff-daca-4a8c-ab36-9400b7532406 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:00.402504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35ccff-daca-4a8c-ab36-9400b7532406 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:00.402770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35ccff-daca-4a8c-ab36-9400b7532406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:00.430604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-9e35ccff-daca-4a8c-ab36-9400b7532406 None None] MFA Rules not processed for user `bef537efe1d144ec96aee78d7acc851c`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:00.431829 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9e35ccff-daca-4a8c-ab36-9400b7532406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:00.500916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1178/2366] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:00 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:00.511945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb4228-0f96-43a0-bfcb-55425051d4d4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:00.513670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-82cb4228-0f96-43a0-bfcb-55425051d4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:00.592196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cb4228-0f96-43a0-bfcb-55425051d4d4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:00.596426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc03b0-fe38-4308-88b0-0aeea26c75f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:00.597015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc03b0-fe38-4308-88b0-0aeea26c75f6 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:00.597291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc03b0-fe38-4308-88b0-0aeea26c75f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:00.597560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc03b0-fe38-4308-88b0-0aeea26c75f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:00.606357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc03b0-fe38-4308-88b0-0aeea26c75f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:00.607815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc03b0-fe38-4308-88b0-0aeea26c75f6 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:00.617228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1189/2367] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:22:00 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:22:00.755107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35ccff-daca-4a8c-ab36-9400b7532406 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:00.817842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35ccff-daca-4a8c-ab36-9400b7532406 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:00.822235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:00.822845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:00.823120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:00.823383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] PATH_INFO: `/v3/users/500ff8ecce8641d6add9602720accd69` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:00.824499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=500ff8ecce8641d6add9602720accd69)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:00.826400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:00.874646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '500ff8ecce8641d6add9602720accd69'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:00.875817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '500ff8ecce8641d6add9602720accd69'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:00.883003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '500ff8ecce8641d6add9602720accd69'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:00.885024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:00.892722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '500ff8ecce8641d6add9602720accd69'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:00.893459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '500ff8ecce8641d6add9602720accd69'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:00.911130 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1179/2368] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:22:00 2022] DELETE /identity/v3/users/500ff8ecce8641d6add9602720accd69 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:00.915641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72551a-1600-42e6-9cef-db95bfc8b7b8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:00.915921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72551a-1600-42e6-9cef-db95bfc8b7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:00.916217 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72551a-1600-42e6-9cef-db95bfc8b7b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:00.934904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ac72551a-1600-42e6-9cef-db95bfc8b7b8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:00.936115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ac72551a-1600-42e6-9cef-db95bfc8b7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:00.995891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1190/2369] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:01.010537 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:01.012166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:01.071456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1b6095-f15c-4391-b98a-80828b64a0ca admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:01.075128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9874c60-bd32-4dd2-a33e-89aa7cb88529 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:01.075757 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9874c60-bd32-4dd2-a33e-89aa7cb88529 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:01.076036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9874c60-bd32-4dd2-a33e-89aa7cb88529 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:01.076298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9874c60-bd32-4dd2-a33e-89aa7cb88529 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:01.086525 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9874c60-bd32-4dd2-a33e-89aa7cb88529 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:01.088050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9874c60-bd32-4dd2-a33e-89aa7cb88529 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:01.097929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1180/2370] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:22:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:22:01.670113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72551a-1600-42e6-9cef-db95bfc8b7b8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:01.671602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ac72551a-1600-42e6-9cef-db95bfc8b7b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:01.689494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72551a-1600-42e6-9cef-db95bfc8b7b8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:01.695172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:01.695963 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:01.696242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:01.696509 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] PATH_INFO: `/v3/projects/a8ec220c0d1d496d8df58807675e8c7c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:01.697976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ec220c0d1d496d8df58807675e8c7c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:01.701464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:01.742721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:INSERT=4,keystone:SELECT=94,keystone:DELETE=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:01.757734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ec220c0d1d496d8df58807675e8c7c', 'request_id': 'req-8568f04a-3337-49c0-b520-e64b5debf353'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:01.763583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ec220c0d1d496d8df58807675e8c7c', 'request_id': 'req-8568f04a-3337-49c0-b520-e64b5debf353'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:01.771208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ec220c0d1d496d8df58807675e8c7c', 'request_id': 'req-8568f04a-3337-49c0-b520-e64b5debf353'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:01.783063 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] The token cache is being invalidate because project a8ec220c0d1d496d8df58807675e8c7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:22:01.783407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:01.784219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:01.785718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1191/2371] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:22:01 2022] DELETE /identity/v3/projects/a8ec220c0d1d496d8df58807675e8c7c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:01.789687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9874c60-bd32-4dd2-a33e-89aa7cb88529 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:01.799852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9874c60-bd32-4dd2-a33e-89aa7cb88529 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:01.857387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:01.858240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:01.858525 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:01.858868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] PATH_INFO: `/v3/users/bef537efe1d144ec96aee78d7acc851c` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:01.860279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bef537efe1d144ec96aee78d7acc851c)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:01.862208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:01.912941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bef537efe1d144ec96aee78d7acc851c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:01.913879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bef537efe1d144ec96aee78d7acc851c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:01.923152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bef537efe1d144ec96aee78d7acc851c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:01.930419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bef537efe1d144ec96aee78d7acc851c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:01.931066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-78f9fec4-f026-4dbf-a33d-abb94d84a207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bef537efe1d144ec96aee78d7acc851c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:01.942756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:01.955765 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1181/2372] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:22:01 2022] DELETE /identity/v3/users/bef537efe1d144ec96aee78d7acc851c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:02.288635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:02.290068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:02.318533 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568f04a-3337-49c0-b520-e64b5debf353 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:02.385634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:02.386289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:02.386571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:02.386843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] PATH_INFO: `/v3/projects/4fc9c98f39c747d8a60e35cc6c4e8039` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:02.388130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc9c98f39c747d8a60e35cc6c4e8039)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:02.390187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:02.455355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc9c98f39c747d8a60e35cc6c4e8039', 'request_id': 'req-75792335-6154-400e-a607-2d94e05f0375'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:02.461555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc9c98f39c747d8a60e35cc6c4e8039', 'request_id': 'req-75792335-6154-400e-a607-2d94e05f0375'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:02.469397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc9c98f39c747d8a60e35cc6c4e8039', 'request_id': 'req-75792335-6154-400e-a607-2d94e05f0375'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:02.481085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] The token cache is being invalidate because project 4fc9c98f39c747d8a60e35cc6c4e8039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:22:02.481383 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:02.482042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-75792335-6154-400e-a607-2d94e05f0375 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:02.483700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1192/2373] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:22:02 2022] DELETE /identity/v3/projects/4fc9c98f39c747d8a60e35cc6c4e8039 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:02.488807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83863f03-03bb-4c54-af58-c5df307cabd8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:02.489178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83863f03-03bb-4c54-af58-c5df307cabd8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:02.489495 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83863f03-03bb-4c54-af58-c5df307cabd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:02.510078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-83863f03-03bb-4c54-af58-c5df307cabd8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:02.511363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-83863f03-03bb-4c54-af58-c5df307cabd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:02.571416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1182/2374] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:02.580712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607083-1381-4c9b-bd90-01a94fb33252 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:02.580997 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607083-1381-4c9b-bd90-01a94fb33252 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:02.581268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2607083-1381-4c9b-bd90-01a94fb33252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:02.601382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d2607083-1381-4c9b-bd90-01a94fb33252 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:02.602576 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d2607083-1381-4c9b-bd90-01a94fb33252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:02.623044 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1193/2375] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:02.627775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83863f03-03bb-4c54-af58-c5df307cabd8 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:02.638662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83863f03-03bb-4c54-af58-c5df307cabd8 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:02.644608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bea37-e09c-4ed9-8e04-79f8b733ffde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:02.646016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bea37-e09c-4ed9-8e04-79f8b733ffde admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:02.646472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bea37-e09c-4ed9-8e04-79f8b733ffde admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:02.646734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70bea37-e09c-4ed9-8e04-79f8b733ffde admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:02.647667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bea37-e09c-4ed9-8e04-79f8b733ffde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:22:02.648076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bea37-e09c-4ed9-8e04-79f8b733ffde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:02.649803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70bea37-e09c-4ed9-8e04-79f8b733ffde admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:02.666138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1183/2376] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:22:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:22:02.666989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607083-1381-4c9b-bd90-01a94fb33252 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:02.676989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2607083-1381-4c9b-bd90-01a94fb33252 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:02.683242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6374a-01eb-40a6-bf39-38ca462b506d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:02.684339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6374a-01eb-40a6-bf39-38ca462b506d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:02.685696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6374a-01eb-40a6-bf39-38ca462b506d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:02.686079 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6374a-01eb-40a6-bf39-38ca462b506d admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:02.686842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6374a-01eb-40a6-bf39-38ca462b506d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:02.690812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6374a-01eb-40a6-bf39-38ca462b506d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:02.726405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1194/2377] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:22:02 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:22:02.727402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bea37-e09c-4ed9-8e04-79f8b733ffde admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:02.738651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70bea37-e09c-4ed9-8e04-79f8b733ffde admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:02.812398 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d255b26-6ce9-445f-bdcb-2140db2a3419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:02.813307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255b26-6ce9-445f-bdcb-2140db2a3419 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:02.813307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255b26-6ce9-445f-bdcb-2140db2a3419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:02.813565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d255b26-6ce9-445f-bdcb-2140db2a3419 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:02.814232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d255b26-6ce9-445f-bdcb-2140db2a3419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:02.816234 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d255b26-6ce9-445f-bdcb-2140db2a3419 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:02.839537 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1184/2378] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:22:02 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:22:02.843787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6374a-01eb-40a6-bf39-38ca462b506d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:02.856593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6374a-01eb-40a6-bf39-38ca462b506d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:02.861747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb897b13-854b-4950-b2d9-beb5d32ce068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:02.862357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb897b13-854b-4950-b2d9-beb5d32ce068 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:02.862635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb897b13-854b-4950-b2d9-beb5d32ce068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:02.863170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb897b13-854b-4950-b2d9-beb5d32ce068 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:02.863869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb897b13-854b-4950-b2d9-beb5d32ce068 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:02.865575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb897b13-854b-4950-b2d9-beb5d32ce068 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:02.876649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1195/2379] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:22:02.881114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d255b26-6ce9-445f-bdcb-2140db2a3419 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:02.896744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d255b26-6ce9-445f-bdcb-2140db2a3419 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:02.901649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5044d5-5e3f-49c7-b1c2-6510c357a8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:02.902383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5044d5-5e3f-49c7-b1c2-6510c357a8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:02.902697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5044d5-5e3f-49c7-b1c2-6510c357a8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:02.903242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5044d5-5e3f-49c7-b1c2-6510c357a8d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:02.904347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5044d5-5e3f-49c7-b1c2-6510c357a8d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:02.906438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5044d5-5e3f-49c7-b1c2-6510c357a8d5 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:02.918070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1185/2380] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:22:02.923347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb897b13-854b-4950-b2d9-beb5d32ce068 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:02.933915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb897b13-854b-4950-b2d9-beb5d32ce068 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:02.938962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11781819-2f45-4b5a-9821-80aaff6e86df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:02.939660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11781819-2f45-4b5a-9821-80aaff6e86df admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:02.939937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11781819-2f45-4b5a-9821-80aaff6e86df admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:02.940207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11781819-2f45-4b5a-9821-80aaff6e86df admin admin] PATH_INFO: `/v3/projects/5c93065b06544856b387d94ecd32ab5c/users/0e0993e0e82948558f8a859da0e44a0e/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:02.949016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:02.957273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11781819-2f45-4b5a-9821-80aaff6e86df admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c93065b06544856b387d94ecd32ab5c, user_id=0e0993e0e82948558f8a859da0e44a0e, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:02.960336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11781819-2f45-4b5a-9821-80aaff6e86df admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:02.976509 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1196/2381] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:22:02 2022] PUT /identity/v3/projects/5c93065b06544856b387d94ecd32ab5c/users/0e0993e0e82948558f8a859da0e44a0e/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:22:02.987651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047fd81-2ff7-4461-b165-cb599095eaf7 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:02.988196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047fd81-2ff7-4461-b165-cb599095eaf7 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:02.988469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a047fd81-2ff7-4461-b165-cb599095eaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:03.016951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a047fd81-2ff7-4461-b165-cb599095eaf7 None None] MFA Rules not processed for user `0e0993e0e82948558f8a859da0e44a0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:03.018171 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a047fd81-2ff7-4461-b165-cb599095eaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:03.066799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:03.086354 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1186/2382] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:03.099040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11781819-2f45-4b5a-9821-80aaff6e86df admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:03.100807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-11781819-2f45-4b5a-9821-80aaff6e86df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:03.161083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11781819-2f45-4b5a-9821-80aaff6e86df admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:03.164745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066d14f-2b7f-48f7-b5d8-d455aa64b4ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:03.165357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066d14f-2b7f-48f7-b5d8-d455aa64b4ec service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:03.165637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066d14f-2b7f-48f7-b5d8-d455aa64b4ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:03.165911 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066d14f-2b7f-48f7-b5d8-d455aa64b4ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:03.174180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066d14f-2b7f-48f7-b5d8-d455aa64b4ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:03.175657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066d14f-2b7f-48f7-b5d8-d455aa64b4ec service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:03.184243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1197/2383] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:22:03 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:22:03.204576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047fd81-2ff7-4461-b165-cb599095eaf7 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:03.265186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a047fd81-2ff7-4461-b165-cb599095eaf7 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:03.270503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:03.271124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:03.271431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:03.271734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] PATH_INFO: `/v3/users/0e0993e0e82948558f8a859da0e44a0e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:03.272845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e0993e0e82948558f8a859da0e44a0e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:03.274767 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:03.320131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e0993e0e82948558f8a859da0e44a0e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:03.320787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e0993e0e82948558f8a859da0e44a0e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:03.327414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e0993e0e82948558f8a859da0e44a0e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:03.335048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e0993e0e82948558f8a859da0e44a0e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:03.335724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e0993e0e82948558f8a859da0e44a0e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:03.353220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1187/2384] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:22:03 2022] DELETE /identity/v3/users/0e0993e0e82948558f8a859da0e44a0e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:03.357652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d5355-08ca-47e0-a19a-dd2fda2c4e3d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:03.357941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d5355-08ca-47e0-a19a-dd2fda2c4e3d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:03.358254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5d5355-08ca-47e0-a19a-dd2fda2c4e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:03.377884 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-dc5d5355-08ca-47e0-a19a-dd2fda2c4e3d None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:03.379094 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-dc5d5355-08ca-47e0-a19a-dd2fda2c4e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:03.441104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1198/2385] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:03.450502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:03.451918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:03.510498 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5484dbf1-3ba8-4703-beba-774972683203 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:03.514328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316815a8-9e15-4132-a7f2-5df3c56edbe8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:03.515047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316815a8-9e15-4132-a7f2-5df3c56edbe8 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:03.515329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316815a8-9e15-4132-a7f2-5df3c56edbe8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:03.515674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316815a8-9e15-4132-a7f2-5df3c56edbe8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:03.524848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316815a8-9e15-4132-a7f2-5df3c56edbe8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:03.526271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316815a8-9e15-4132-a7f2-5df3c56edbe8 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:03.535825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1188/2386] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:22:03.807980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d5355-08ca-47e0-a19a-dd2fda2c4e3d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:03.809504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-dc5d5355-08ca-47e0-a19a-dd2fda2c4e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:03.821486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5d5355-08ca-47e0-a19a-dd2fda2c4e3d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:03.826103 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:03.826733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:03.827009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:03.827277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] PATH_INFO: `/v3/projects/5c93065b06544856b387d94ecd32ab5c` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:03.828991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c93065b06544856b387d94ecd32ab5c)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:03.831025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:03.880933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c93065b06544856b387d94ecd32ab5c', 'request_id': 'req-bc659aab-4d41-497a-9092-1df8b1e00bc8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:03.886398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c93065b06544856b387d94ecd32ab5c', 'request_id': 'req-bc659aab-4d41-497a-9092-1df8b1e00bc8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:03.894862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c93065b06544856b387d94ecd32ab5c', 'request_id': 'req-bc659aab-4d41-497a-9092-1df8b1e00bc8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:03.908403 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] The token cache is being invalidate because project 5c93065b06544856b387d94ecd32ab5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:22:03.908697 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:03.909342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-bc659aab-4d41-497a-9092-1df8b1e00bc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:03.910835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1199/2387] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:22:03 2022] DELETE /identity/v3/projects/5c93065b06544856b387d94ecd32ab5c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:03.915706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c769c65-b390-4c84-a54b-aa8554e2b7d5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:03.916232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c769c65-b390-4c84-a54b-aa8554e2b7d5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:03.916330 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c769c65-b390-4c84-a54b-aa8554e2b7d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:03.937304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7c769c65-b390-4c84-a54b-aa8554e2b7d5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:03.939103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7c769c65-b390-4c84-a54b-aa8554e2b7d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:03.948039 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:04.015062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1189/2388] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:04.024288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc376f42-2fcf-44eb-82a0-fac830322bf5 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.024571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc376f42-2fcf-44eb-82a0-fac830322bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.024843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc376f42-2fcf-44eb-82a0-fac830322bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.044644 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-cc376f42-2fcf-44eb-82a0-fac830322bf5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:04.045846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cc376f42-2fcf-44eb-82a0-fac830322bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:04.054540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1200/2389] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:04.058734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c769c65-b390-4c84-a54b-aa8554e2b7d5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:04.069384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c769c65-b390-4c84-a54b-aa8554e2b7d5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:04.074012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa74f8-0d55-4b66-baef-401145a8d483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:04.074675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa74f8-0d55-4b66-baef-401145a8d483 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.074951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa74f8-0d55-4b66-baef-401145a8d483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.075302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fa74f8-0d55-4b66-baef-401145a8d483 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.076057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa74f8-0d55-4b66-baef-401145a8d483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:22:04.076418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa74f8-0d55-4b66-baef-401145a8d483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:04.078011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fa74f8-0d55-4b66-baef-401145a8d483 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:04.094878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1190/2390] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:22:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:22:04.095983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc376f42-2fcf-44eb-82a0-fac830322bf5 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:04.105864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc376f42-2fcf-44eb-82a0-fac830322bf5 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:04.111002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d09599-e79f-4ccc-b2b2-3bf6137bb0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:04.111652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d09599-e79f-4ccc-b2b2-3bf6137bb0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.111926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d09599-e79f-4ccc-b2b2-3bf6137bb0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.112190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d09599-e79f-4ccc-b2b2-3bf6137bb0d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.113125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d09599-e79f-4ccc-b2b2-3bf6137bb0d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:04.115025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d09599-e79f-4ccc-b2b2-3bf6137bb0d6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:04.119763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:04.148830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1201/2391] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:22:04 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:22:04.152520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa74f8-0d55-4b66-baef-401145a8d483 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:04.164059 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fa74f8-0d55-4b66-baef-401145a8d483 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:04.222426 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c2164-a1f5-49af-83d5-2b669991878c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:04.223031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2164-a1f5-49af-83d5-2b669991878c admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.223307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2164-a1f5-49af-83d5-2b669991878c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.223604 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c2164-a1f5-49af-83d5-2b669991878c admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.224222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c2164-a1f5-49af-83d5-2b669991878c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:04.226159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c2164-a1f5-49af-83d5-2b669991878c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:04.249532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1191/2392] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:22:04 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:22:04.254304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d09599-e79f-4ccc-b2b2-3bf6137bb0d6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:04.264597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d09599-e79f-4ccc-b2b2-3bf6137bb0d6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:04.269961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc40fbd-8775-4a40-800c-289f61e7af4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:04.270589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc40fbd-8775-4a40-800c-289f61e7af4b admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.270857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc40fbd-8775-4a40-800c-289f61e7af4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.271148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc40fbd-8775-4a40-800c-289f61e7af4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.271998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc40fbd-8775-4a40-800c-289f61e7af4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:04.273991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc40fbd-8775-4a40-800c-289f61e7af4b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:04.285359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1202/2393] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:22:04.289461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c2164-a1f5-49af-83d5-2b669991878c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:04.299898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c2164-a1f5-49af-83d5-2b669991878c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:04.304574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed142fb-d362-4205-be68-80afc830aa42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:04.305172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed142fb-d362-4205-be68-80afc830aa42 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.305445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed142fb-d362-4205-be68-80afc830aa42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.305711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed142fb-d362-4205-be68-80afc830aa42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.306347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed142fb-d362-4205-be68-80afc830aa42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:04.307972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed142fb-d362-4205-be68-80afc830aa42 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:04.318791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1192/2394] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:22:04.323393 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc40fbd-8775-4a40-800c-289f61e7af4b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:04.334767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc40fbd-8775-4a40-800c-289f61e7af4b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:04.339369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823dba3-b59b-452c-8f40-8fed963aae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:04.340192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823dba3-b59b-452c-8f40-8fed963aae07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.340463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823dba3-b59b-452c-8f40-8fed963aae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.340731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823dba3-b59b-452c-8f40-8fed963aae07 admin admin] PATH_INFO: `/v3/projects/c2ccbd606a774edab4bed59416cd0c70/users/985420b8088c41a7a71c771cde4ffecd/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.357827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823dba3-b59b-452c-8f40-8fed963aae07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ccbd606a774edab4bed59416cd0c70, user_id=985420b8088c41a7a71c771cde4ffecd, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:04.361735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823dba3-b59b-452c-8f40-8fed963aae07 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:04.375946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1203/2395] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:22:04 2022] PUT /identity/v3/projects/c2ccbd606a774edab4bed59416cd0c70/users/985420b8088c41a7a71c771cde4ffecd/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:04.387321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de6a3-e0fb-4026-9159-249f09342cc0 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.387559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de6a3-e0fb-4026-9159-249f09342cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.387834 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32de6a3-e0fb-4026-9159-249f09342cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.416839 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c32de6a3-e0fb-4026-9159-249f09342cc0 None None] MFA Rules not processed for user `985420b8088c41a7a71c771cde4ffecd`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:04.418056 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c32de6a3-e0fb-4026-9159-249f09342cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:04.491737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1193/2396] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:04.503982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823dba3-b59b-452c-8f40-8fed963aae07 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:04.505939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1823dba3-b59b-452c-8f40-8fed963aae07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:04.567035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823dba3-b59b-452c-8f40-8fed963aae07 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:04.570714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dedfae7-ba4f-45f1-877b-645f0c827422 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:04.571320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedfae7-ba4f-45f1-877b-645f0c827422 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.571655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedfae7-ba4f-45f1-877b-645f0c827422 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.571926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dedfae7-ba4f-45f1-877b-645f0c827422 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.581602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedfae7-ba4f-45f1-877b-645f0c827422 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:04.583085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dedfae7-ba4f-45f1-877b-645f0c827422 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:04.592585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1204/2397] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:22:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:22:04.617216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32de6a3-e0fb-4026-9159-249f09342cc0 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:04.628810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:04.700494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32de6a3-e0fb-4026-9159-249f09342cc0 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:04.705218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:04.705852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.706127 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.706396 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] PATH_INFO: `/v3/users/985420b8088c41a7a71c771cde4ffecd` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.707560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] RBAC: Authorizing `identity:delete_user(user_id=985420b8088c41a7a71c771cde4ffecd)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:04.709720 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:04.764550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '985420b8088c41a7a71c771cde4ffecd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:04.765198 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '985420b8088c41a7a71c771cde4ffecd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:04.773552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '985420b8088c41a7a71c771cde4ffecd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:04.782232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '985420b8088c41a7a71c771cde4ffecd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:04.783029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '985420b8088c41a7a71c771cde4ffecd'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:04.804368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1194/2398] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:22:04 2022] DELETE /identity/v3/users/985420b8088c41a7a71c771cde4ffecd => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:04.809049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1416717-b60b-4c23-abfe-7de8d7c40c02 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.809526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1416717-b60b-4c23-abfe-7de8d7c40c02 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.809682 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1416717-b60b-4c23-abfe-7de8d7c40c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.830484 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c1416717-b60b-4c23-abfe-7de8d7c40c02 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:04.832262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c1416717-b60b-4c23-abfe-7de8d7c40c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:04.896781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1205/2399] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:22:04.907491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:04.908931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:04.971420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5ef2dc-6bf2-4cc9-a39f-a64ec869a452 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:04.975400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7ddf00-e93e-4e7f-92cb-cc4f3826377b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:04.976231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ddf00-e93e-4e7f-92cb-cc4f3826377b service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:04.976510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ddf00-e93e-4e7f-92cb-cc4f3826377b service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:04.976783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7ddf00-e93e-4e7f-92cb-cc4f3826377b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:04.987033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ddf00-e93e-4e7f-92cb-cc4f3826377b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:04.988772 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7ddf00-e93e-4e7f-92cb-cc4f3826377b service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:04.998661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1195/2400] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:22:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:22:05.414261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1416717-b60b-4c23-abfe-7de8d7c40c02 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:05.415804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c1416717-b60b-4c23-abfe-7de8d7c40c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:05.445069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1416717-b60b-4c23-abfe-7de8d7c40c02 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:05.450417 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:05.451468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:05.451760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:05.452131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] PATH_INFO: `/v3/projects/c2ccbd606a774edab4bed59416cd0c70` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:05.453857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ccbd606a774edab4bed59416cd0c70)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:05.455948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:05.505017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:05.506790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ccbd606a774edab4bed59416cd0c70', 'request_id': 'req-25c7d676-692f-4158-9062-fcd67008d48d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:05.514095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ccbd606a774edab4bed59416cd0c70', 'request_id': 'req-25c7d676-692f-4158-9062-fcd67008d48d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:05.522002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ccbd606a774edab4bed59416cd0c70', 'request_id': 'req-25c7d676-692f-4158-9062-fcd67008d48d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:05.533835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] The token cache is being invalidate because project c2ccbd606a774edab4bed59416cd0c70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:22:05.534199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:05.534872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-25c7d676-692f-4158-9062-fcd67008d48d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:05.536397 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1206/2401] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:22:05 2022] DELETE /identity/v3/projects/c2ccbd606a774edab4bed59416cd0c70 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:05.541516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52dd90-68f7-4ba1-ab9f-666591ac2397 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:05.541874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52dd90-68f7-4ba1-ab9f-666591ac2397 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:05.542155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52dd90-68f7-4ba1-ab9f-666591ac2397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:05.562763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-fa52dd90-68f7-4ba1-ab9f-666591ac2397 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:05.564010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-fa52dd90-68f7-4ba1-ab9f-666591ac2397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:05.590723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:05.627329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1196/2402] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:05.635243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f757c1-a7aa-4dff-b47d-ffa62d4f32ec None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:05.635560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f757c1-a7aa-4dff-b47d-ffa62d4f32ec None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:05.635878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f757c1-a7aa-4dff-b47d-ffa62d4f32ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:05.655477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-23f757c1-a7aa-4dff-b47d-ffa62d4f32ec None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:05.657252 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-23f757c1-a7aa-4dff-b47d-ffa62d4f32ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:05.665395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1207/2403] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:05.669483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52dd90-68f7-4ba1-ab9f-666591ac2397 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:05.680004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52dd90-68f7-4ba1-ab9f-666591ac2397 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:05.684710 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28446ec-d88e-4227-ad31-48be727edb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:05.685313 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28446ec-d88e-4227-ad31-48be727edb61 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:05.685594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28446ec-d88e-4227-ad31-48be727edb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:05.685865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28446ec-d88e-4227-ad31-48be727edb61 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:05.686507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28446ec-d88e-4227-ad31-48be727edb61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:22:05.686807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28446ec-d88e-4227-ad31-48be727edb61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:05.689150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28446ec-d88e-4227-ad31-48be727edb61 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:05.705452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1197/2404] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:22:05.707145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f757c1-a7aa-4dff-b47d-ffa62d4f32ec None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:05.717487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f757c1-a7aa-4dff-b47d-ffa62d4f32ec None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:05.723013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eeef0d0-06ca-4be8-be2f-eae961979c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:05.723637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef0d0-06ca-4be8-be2f-eae961979c86 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:05.723915 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef0d0-06ca-4be8-be2f-eae961979c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:05.724185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeef0d0-06ca-4be8-be2f-eae961979c86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:05.724827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeef0d0-06ca-4be8-be2f-eae961979c86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:05.726741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eeef0d0-06ca-4be8-be2f-eae961979c86 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:05.757319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1208/2405] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:22:05 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:22:05.761713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28446ec-d88e-4227-ad31-48be727edb61 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:05.772758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28446ec-d88e-4227-ad31-48be727edb61 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:05.836560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8450d4-b182-4bbf-9a7a-c28074883a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:05.837172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8450d4-b182-4bbf-9a7a-c28074883a0b admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:05.837449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8450d4-b182-4bbf-9a7a-c28074883a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:05.837714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8450d4-b182-4bbf-9a7a-c28074883a0b admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:05.838354 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8450d4-b182-4bbf-9a7a-c28074883a0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:05.840336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8450d4-b182-4bbf-9a7a-c28074883a0b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:05.844773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a42562-484b-4308-9fe3-7db788122f47 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:05.845281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a42562-484b-4308-9fe3-7db788122f47 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:05.845365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a42562-484b-4308-9fe3-7db788122f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:05.866318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1198/2406] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:22:05 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:22:05.866520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-27a42562-484b-4308-9fe3-7db788122f47 None None] MFA Rules not processed for user `db2f1171c5394a1c8879cd09c1e685b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:05.867795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-27a42562-484b-4308-9fe3-7db788122f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:05.870429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8450d4-b182-4bbf-9a7a-c28074883a0b admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:05.880721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8450d4-b182-4bbf-9a7a-c28074883a0b admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:05.885366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62e33b-9973-4cd6-ad7a-6235e30ea164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:05.885972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62e33b-9973-4cd6-ad7a-6235e30ea164 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:05.886268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62e33b-9973-4cd6-ad7a-6235e30ea164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:05.886573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee62e33b-9973-4cd6-ad7a-6235e30ea164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:05.887172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62e33b-9973-4cd6-ad7a-6235e30ea164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:05.888801 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee62e33b-9973-4cd6-ad7a-6235e30ea164 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:05.901749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1199/2407] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:22:05.902946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62e33b-9973-4cd6-ad7a-6235e30ea164 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:05.912812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee62e33b-9973-4cd6-ad7a-6235e30ea164 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:05.917383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c375bdfd-6e45-46d0-8e4c-1e568f8e8d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:05.917976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375bdfd-6e45-46d0-8e4c-1e568f8e8d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:05.918248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375bdfd-6e45-46d0-8e4c-1e568f8e8d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:05.918516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c375bdfd-6e45-46d0-8e4c-1e568f8e8d19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:05.919142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375bdfd-6e45-46d0-8e4c-1e568f8e8d19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:05.920787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c375bdfd-6e45-46d0-8e4c-1e568f8e8d19 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:05.927843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1209/2408] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:22:05.935421 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1200/2409] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:22:05.936200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a42562-484b-4308-9fe3-7db788122f47 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:05.947590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a42562-484b-4308-9fe3-7db788122f47 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:05.952286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd791cc-6446-4dc7-b0b3-a80fe9d56dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:05.952936 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd791cc-6446-4dc7-b0b3-a80fe9d56dab admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:05.953832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd791cc-6446-4dc7-b0b3-a80fe9d56dab admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:05.954109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd791cc-6446-4dc7-b0b3-a80fe9d56dab admin admin] PATH_INFO: `/v3/projects/dc0812b1b76548e88f10ad68e8d4a1e9/users/223c6cbab41a4ca2969b24cb5603cbca/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:05.972204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd791cc-6446-4dc7-b0b3-a80fe9d56dab admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc0812b1b76548e88f10ad68e8d4a1e9, user_id=223c6cbab41a4ca2969b24cb5603cbca, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:05.975641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd791cc-6446-4dc7-b0b3-a80fe9d56dab admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:05.989584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1210/2410] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:22:05 2022] PUT /identity/v3/projects/dc0812b1b76548e88f10ad68e8d4a1e9/users/223c6cbab41a4ca2969b24cb5603cbca/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:22:06.000375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de00db-b548-44ce-bd4b-da0f659a1854 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:06.000906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de00db-b548-44ce-bd4b-da0f659a1854 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:06.001063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de00db-b548-44ce-bd4b-da0f659a1854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:06.029922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-81de00db-b548-44ce-bd4b-da0f659a1854 None None] MFA Rules not processed for user `223c6cbab41a4ca2969b24cb5603cbca`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:06.031129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-81de00db-b548-44ce-bd4b-da0f659a1854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:06.097813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1201/2411] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:22:06.109092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd791cc-6446-4dc7-b0b3-a80fe9d56dab admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:06.111001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3cd791cc-6446-4dc7-b0b3-a80fe9d56dab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:06.169438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd791cc-6446-4dc7-b0b3-a80fe9d56dab admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:06.173160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98959ad9-9f4b-4c43-97cd-d28573b88741 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:06.173762 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98959ad9-9f4b-4c43-97cd-d28573b88741 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:06.174036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98959ad9-9f4b-4c43-97cd-d28573b88741 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:06.174307 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98959ad9-9f4b-4c43-97cd-d28573b88741 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:06.182996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98959ad9-9f4b-4c43-97cd-d28573b88741 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:06.184482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98959ad9-9f4b-4c43-97cd-d28573b88741 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:06.194500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1211/2412] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:22:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:22:16.094843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:16.190319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=80,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:31.100970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de00db-b548-44ce-bd4b-da0f659a1854 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:31.163054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de00db-b548-44ce-bd4b-da0f659a1854 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:31.167608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:31.168230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:31.168504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:31.168780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] PATH_INFO: `/v3/users/223c6cbab41a4ca2969b24cb5603cbca` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:31.169878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=223c6cbab41a4ca2969b24cb5603cbca)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:31.171818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:31.217847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '223c6cbab41a4ca2969b24cb5603cbca'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:31.218487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '223c6cbab41a4ca2969b24cb5603cbca'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:31.225982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '223c6cbab41a4ca2969b24cb5603cbca'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:31.233246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '223c6cbab41a4ca2969b24cb5603cbca'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:31.233942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '223c6cbab41a4ca2969b24cb5603cbca'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:31.257190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1202/2413] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:22:31 2022] DELETE /identity/v3/users/223c6cbab41a4ca2969b24cb5603cbca => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:31.261603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d771753-2dfd-4cd5-b20a-def952b0bea0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:31.262097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d771753-2dfd-4cd5-b20a-def952b0bea0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:31.262207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d771753-2dfd-4cd5-b20a-def952b0bea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:31.281463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-7d771753-2dfd-4cd5-b20a-def952b0bea0 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:31.282882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7d771753-2dfd-4cd5-b20a-def952b0bea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:31.340803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1212/2414] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:31.350970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:31.352400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:31.408734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dab775-b100-48b6-9169-702134c27c9c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:31.412442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561b314d-7d07-4d3d-8159-2b8517337147 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:31.413045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b314d-7d07-4d3d-8159-2b8517337147 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:31.413323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b314d-7d07-4d3d-8159-2b8517337147 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:31.413592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561b314d-7d07-4d3d-8159-2b8517337147 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:31.423261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b314d-7d07-4d3d-8159-2b8517337147 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:31.424673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561b314d-7d07-4d3d-8159-2b8517337147 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:31.434681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1203/2415] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:22:31.699860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d771753-2dfd-4cd5-b20a-def952b0bea0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:31.701342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7d771753-2dfd-4cd5-b20a-def952b0bea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:31.713615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d771753-2dfd-4cd5-b20a-def952b0bea0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:31.718687 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:31.720163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:31.720612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:31.720917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] PATH_INFO: `/v3/projects/dc0812b1b76548e88f10ad68e8d4a1e9` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:31.722098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc0812b1b76548e88f10ad68e8d4a1e9)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:31.724099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:31.770869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc0812b1b76548e88f10ad68e8d4a1e9', 'request_id': 'req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:31.776430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc0812b1b76548e88f10ad68e8d4a1e9', 'request_id': 'req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:31.784307 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc0812b1b76548e88f10ad68e8d4a1e9', 'request_id': 'req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:31.795868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] The token cache is being invalidate because project dc0812b1b76548e88f10ad68e8d4a1e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:22:31.796153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:31.796800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b2eb18c1-c3c7-4e86-b5af-e165f702caf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:22:31.799118 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1213/2416] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:22:31 2022] DELETE /identity/v3/projects/dc0812b1b76548e88f10ad68e8d4a1e9 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:22:31.804802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634729b5-033d-4797-96b0-645d1c92e3e8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:31.805185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634729b5-033d-4797-96b0-645d1c92e3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:31.805524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634729b5-033d-4797-96b0-645d1c92e3e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:31.824866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-634729b5-033d-4797-96b0-645d1c92e3e8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:31.826152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-634729b5-033d-4797-96b0-645d1c92e3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:31.843742 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:31.889502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1204/2417] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:31.897093 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b126-4e20-4ca4-907f-0602bffa0515 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:31.897452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b126-4e20-4ca4-907f-0602bffa0515 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:31.897791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8431b126-4e20-4ca4-907f-0602bffa0515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:31.921235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8431b126-4e20-4ca4-907f-0602bffa0515 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:31.923399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8431b126-4e20-4ca4-907f-0602bffa0515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:31.932470 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1214/2418] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:31.936852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634729b5-033d-4797-96b0-645d1c92e3e8 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:31.946635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634729b5-033d-4797-96b0-645d1c92e3e8 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:31.951358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a300e-a3a1-47cb-88a3-d6dab349a222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:31.951990 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a300e-a3a1-47cb-88a3-d6dab349a222 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:31.952265 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a300e-a3a1-47cb-88a3-d6dab349a222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:31.952534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a300e-a3a1-47cb-88a3-d6dab349a222 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:31.953171 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a300e-a3a1-47cb-88a3-d6dab349a222 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:22:31.954024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a300e-a3a1-47cb-88a3-d6dab349a222 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:31.955632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4a300e-a3a1-47cb-88a3-d6dab349a222 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:31.967205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1205/2419] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:22:31.971681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b126-4e20-4ca4-907f-0602bffa0515 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:31.981095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8431b126-4e20-4ca4-907f-0602bffa0515 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:31.986728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6524c9b-32e4-4a5e-b35f-80943fc77f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:31.987346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6524c9b-32e4-4a5e-b35f-80943fc77f50 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:31.987658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6524c9b-32e4-4a5e-b35f-80943fc77f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:31.987927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6524c9b-32e4-4a5e-b35f-80943fc77f50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:31.988847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6524c9b-32e4-4a5e-b35f-80943fc77f50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:31.990740 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6524c9b-32e4-4a5e-b35f-80943fc77f50 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:32.017577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1215/2420] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:22:31 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:22:32.021685 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a300e-a3a1-47cb-88a3-d6dab349a222 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:32.031941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a300e-a3a1-47cb-88a3-d6dab349a222 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:32.086947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62050da8-acab-444a-9a96-ac0f860d2cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:32.087570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62050da8-acab-444a-9a96-ac0f860d2cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:32.087845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62050da8-acab-444a-9a96-ac0f860d2cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:32.088228 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62050da8-acab-444a-9a96-ac0f860d2cb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:32.088876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62050da8-acab-444a-9a96-ac0f860d2cb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:32.090796 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62050da8-acab-444a-9a96-ac0f860d2cb9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:32.111896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1206/2421] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:22:32 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:22:32.116291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6524c9b-32e4-4a5e-b35f-80943fc77f50 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:32.127248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6524c9b-32e4-4a5e-b35f-80943fc77f50 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:32.131965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98e393-1d30-4df8-a55a-02fb6c676ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:32.132552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e393-1d30-4df8-a55a-02fb6c676ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:32.132827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e393-1d30-4df8-a55a-02fb6c676ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:32.133095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98e393-1d30-4df8-a55a-02fb6c676ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:32.133742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98e393-1d30-4df8-a55a-02fb6c676ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:32.135374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be98e393-1d30-4df8-a55a-02fb6c676ef2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:32.145620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1216/2422] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:22:32.149721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62050da8-acab-444a-9a96-ac0f860d2cb9 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:32.160011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62050da8-acab-444a-9a96-ac0f860d2cb9 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:32.164639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be81f3-7a6a-4515-aa34-cfcc1eb2556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:32.165223 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be81f3-7a6a-4515-aa34-cfcc1eb2556a admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:32.165496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be81f3-7a6a-4515-aa34-cfcc1eb2556a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:32.165763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be81f3-7a6a-4515-aa34-cfcc1eb2556a admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:32.166443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be81f3-7a6a-4515-aa34-cfcc1eb2556a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:32.168097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be81f3-7a6a-4515-aa34-cfcc1eb2556a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:32.179053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1207/2423] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:22:32.183005 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98e393-1d30-4df8-a55a-02fb6c676ef2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:32.193398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98e393-1d30-4df8-a55a-02fb6c676ef2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:32.197941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c7f88-7ab6-4b15-bedc-e419400b2482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:32.198571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c7f88-7ab6-4b15-bedc-e419400b2482 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:32.198847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c7f88-7ab6-4b15-bedc-e419400b2482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:32.199116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c7f88-7ab6-4b15-bedc-e419400b2482 admin admin] PATH_INFO: `/v3/projects/c8d41b873ac94a33beb63a2d89a218ff/users/f4638a1936f5467db502bda82320b113/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:32.213822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c7f88-7ab6-4b15-bedc-e419400b2482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8d41b873ac94a33beb63a2d89a218ff, user_id=f4638a1936f5467db502bda82320b113, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:32.216601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c7f88-7ab6-4b15-bedc-e419400b2482 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:32.229891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1217/2424] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:22:32 2022] PUT /identity/v3/projects/c8d41b873ac94a33beb63a2d89a218ff/users/f4638a1936f5467db502bda82320b113/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:32.234987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71718819-1f31-4ecf-ae3e-cf8de47f5465 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:32.235275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71718819-1f31-4ecf-ae3e-cf8de47f5465 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:32.235565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71718819-1f31-4ecf-ae3e-cf8de47f5465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:32.254707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-71718819-1f31-4ecf-ae3e-cf8de47f5465 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:32.255942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-71718819-1f31-4ecf-ae3e-cf8de47f5465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:32.314018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1208/2425] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:32.323531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c7f88-7ab6-4b15-bedc-e419400b2482 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:32.325299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-134c7f88-7ab6-4b15-bedc-e419400b2482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:32.347860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:32.384600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c7f88-7ab6-4b15-bedc-e419400b2482 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:32.388411 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fba570-f1a4-42ed-930e-12118e459e00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:32.389014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fba570-f1a4-42ed-930e-12118e459e00 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:32.389289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fba570-f1a4-42ed-930e-12118e459e00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:32.389590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fba570-f1a4-42ed-930e-12118e459e00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:32.399307 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fba570-f1a4-42ed-930e-12118e459e00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:32.400787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fba570-f1a4-42ed-930e-12118e459e00 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:32.410197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1218/2426] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:22:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:22:36.592191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e3705-e9f0-4870-a4bc-635ec8af74b9 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:36.592773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e3705-e9f0-4870-a4bc-635ec8af74b9 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:36.592773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e3705-e9f0-4870-a4bc-635ec8af74b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:36.629943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-837e3705-e9f0-4870-a4bc-635ec8af74b9 None None] MFA Rules not processed for user `f4638a1936f5467db502bda82320b113`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:36.631251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-837e3705-e9f0-4870-a4bc-635ec8af74b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:36.650128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:36.706953 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1209/2427] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:36 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:36.710462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fba570-f1a4-42ed-930e-12118e459e00 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:36.729969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fba570-f1a4-42ed-930e-12118e459e00 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:36.749841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5f94b-457b-4d7d-87e9-12fdbd045956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:36.751258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5f94b-457b-4d7d-87e9-12fdbd045956 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:36.751626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5f94b-457b-4d7d-87e9-12fdbd045956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:36.751921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5f94b-457b-4d7d-87e9-12fdbd045956 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:36.752549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5f94b-457b-4d7d-87e9-12fdbd045956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:36.754470 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5f94b-457b-4d7d-87e9-12fdbd045956 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:36.791776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1219/2428] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:22:36 2022] POST /identity/v3/projects => generated 379 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:22:36.792941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e3705-e9f0-4870-a4bc-635ec8af74b9 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:36.805428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e3705-e9f0-4870-a4bc-635ec8af74b9 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:36.873672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19902e02-3415-49b1-aa54-0cc66f9e3365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:36.875106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19902e02-3415-49b1-aa54-0cc66f9e3365 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:36.876017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19902e02-3415-49b1-aa54-0cc66f9e3365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:36.876523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19902e02-3415-49b1-aa54-0cc66f9e3365 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:36.877360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19902e02-3415-49b1-aa54-0cc66f9e3365 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:36.879763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19902e02-3415-49b1-aa54-0cc66f9e3365 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:36.905564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1210/2429] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:22:36 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:22:36.909526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5f94b-457b-4d7d-87e9-12fdbd045956 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:36.920339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5f94b-457b-4d7d-87e9-12fdbd045956 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:36.925806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe1b22-4a50-482c-8478-45c783d65fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:36.926426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe1b22-4a50-482c-8478-45c783d65fbc admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:36.926708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe1b22-4a50-482c-8478-45c783d65fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:36.926988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abe1b22-4a50-482c-8478-45c783d65fbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:36.927702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe1b22-4a50-482c-8478-45c783d65fbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:36.929432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abe1b22-4a50-482c-8478-45c783d65fbc admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:36.941563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1220/2430] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:22:36.945777 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19902e02-3415-49b1-aa54-0cc66f9e3365 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:36.957276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19902e02-3415-49b1-aa54-0cc66f9e3365 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:36.962048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745e604-dd7d-4f7f-9e4a-3ccd55e86b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:36.962716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745e604-dd7d-4f7f-9e4a-3ccd55e86b90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:36.962992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745e604-dd7d-4f7f-9e4a-3ccd55e86b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:36.963262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b745e604-dd7d-4f7f-9e4a-3ccd55e86b90 admin admin] PATH_INFO: `/v3/projects/0aa91e4e0c684befa53d7f0fc50ac25a/users/26c4e1d1e313497cbfa63825c59c724d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:36.979160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745e604-dd7d-4f7f-9e4a-3ccd55e86b90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa91e4e0c684befa53d7f0fc50ac25a, user_id=26c4e1d1e313497cbfa63825c59c724d, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:36.981815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b745e604-dd7d-4f7f-9e4a-3ccd55e86b90 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:36.996576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1211/2431] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:22:36 2022] PUT /identity/v3/projects/0aa91e4e0c684befa53d7f0fc50ac25a/users/26c4e1d1e313497cbfa63825c59c724d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:37.001228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe1b22-4a50-482c-8478-45c783d65fbc admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:37.011780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abe1b22-4a50-482c-8478-45c783d65fbc admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:37.074597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634448d9-3e66-4f3e-ad4a-1e31bde5b29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:37.075144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634448d9-3e66-4f3e-ad4a-1e31bde5b29d admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:37.075422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634448d9-3e66-4f3e-ad4a-1e31bde5b29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:37.075725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634448d9-3e66-4f3e-ad4a-1e31bde5b29d admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:37.076372 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634448d9-3e66-4f3e-ad4a-1e31bde5b29d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:37.078024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634448d9-3e66-4f3e-ad4a-1e31bde5b29d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:37.089677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1221/2432] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:22:37.093686 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745e604-dd7d-4f7f-9e4a-3ccd55e86b90 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:37.105334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b745e604-dd7d-4f7f-9e4a-3ccd55e86b90 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:37.110820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5a57a-c869-4566-8cad-e490fed9cdce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:37.111639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5a57a-c869-4566-8cad-e490fed9cdce admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:37.112106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5a57a-c869-4566-8cad-e490fed9cdce admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:37.112392 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a5a57a-c869-4566-8cad-e490fed9cdce admin admin] PATH_INFO: `/v3/projects/0aa91e4e0c684befa53d7f0fc50ac25a/users/26c4e1d1e313497cbfa63825c59c724d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:37.114728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5a57a-c869-4566-8cad-e490fed9cdce admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aa91e4e0c684befa53d7f0fc50ac25a, user_id=26c4e1d1e313497cbfa63825c59c724d, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:37.117559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a5a57a-c869-4566-8cad-e490fed9cdce admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:37.132809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1212/2433] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:22:37 2022] PUT /identity/v3/projects/0aa91e4e0c684befa53d7f0fc50ac25a/users/26c4e1d1e313497cbfa63825c59c724d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:22:41.936297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbd2b8-129e-44d4-92f1-31430d61b393 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:41.936838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbd2b8-129e-44d4-92f1-31430d61b393 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:41.936838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dbd2b8-129e-44d4-92f1-31430d61b393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:41.977014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-97dbd2b8-129e-44d4-92f1-31430d61b393 None None] MFA Rules not processed for user `26c4e1d1e313497cbfa63825c59c724d`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:41.978487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-97dbd2b8-129e-44d4-92f1-31430d61b393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:42.059743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1222/2434] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:22:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:42.073607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5a57a-c869-4566-8cad-e490fed9cdce admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:42.075294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-02a5a57a-c869-4566-8cad-e490fed9cdce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:42.152859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a5a57a-c869-4566-8cad-e490fed9cdce admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:42.158431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0cd53-2680-478d-94f5-99b4001f8d48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:42.159037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0cd53-2680-478d-94f5-99b4001f8d48 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:42.159313 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0cd53-2680-478d-94f5-99b4001f8d48 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:42.159620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0cd53-2680-478d-94f5-99b4001f8d48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:42.165859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:42.224311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0cd53-2680-478d-94f5-99b4001f8d48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:42.225654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0cd53-2680-478d-94f5-99b4001f8d48 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:42.239716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1213/2435] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:22:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:22:43.691615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e950226-64af-44dc-afd0-9f6b3a1b0f4f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:43.692194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e950226-64af-44dc-afd0-9f6b3a1b0f4f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:43.692194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e950226-64af-44dc-afd0-9f6b3a1b0f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:43.718404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8e950226-64af-44dc-afd0-9f6b3a1b0f4f None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:43.721862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8e950226-64af-44dc-afd0-9f6b3a1b0f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:43.730367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:43.821404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1223/2436] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:22:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:43.896376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0cd53-2680-478d-94f5-99b4001f8d48 service nova] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:43.899027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ccf0cd53-2680-478d-94f5-99b4001f8d48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:44.006215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0cd53-2680-478d-94f5-99b4001f8d48 service nova] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:44.011847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b828dee-1965-402c-b3b5-8630b19eaf36 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:44.012608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b828dee-1965-402c-b3b5-8630b19eaf36 service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:44.012948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b828dee-1965-402c-b3b5-8630b19eaf36 service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:44.013290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b828dee-1965-402c-b3b5-8630b19eaf36 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:44.022468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b828dee-1965-402c-b3b5-8630b19eaf36 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:44.024204 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b828dee-1965-402c-b3b5-8630b19eaf36 service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:44.034371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1214/2437] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:22:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:22:53.803880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=20 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:54.029768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=56 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:22:56.619163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb74cd-6239-4260-89f0-36b71fa3ded0 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:56.619720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb74cd-6239-4260-89f0-36b71fa3ded0 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:56.621456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bb74cd-6239-4260-89f0-36b71fa3ded0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:56.643548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-12bb74cd-6239-4260-89f0-36b71fa3ded0 None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:22:56.644762 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-12bb74cd-6239-4260-89f0-36b71fa3ded0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:22:56.657512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1224/2438] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 10 14:22:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:22:56.665712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b828dee-1965-402c-b3b5-8630b19eaf36 service swift] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:22:56.677248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b828dee-1965-402c-b3b5-8630b19eaf36 service swift] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:22:56.682886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966837de-b9b7-4de4-b630-608d34cbaed3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:22:56.683594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966837de-b9b7-4de4-b630-608d34cbaed3 service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:22:56.683959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966837de-b9b7-4de4-b630-608d34cbaed3 service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:22:56.684302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966837de-b9b7-4de4-b630-608d34cbaed3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:22:56.693893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966837de-b9b7-4de4-b630-608d34cbaed3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:22:56.695643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966837de-b9b7-4de4-b630-608d34cbaed3 service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:22:56.705933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1215/2439] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:22:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 14:23:00.760287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb74cd-6239-4260-89f0-36b71fa3ded0 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:00.762012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-12bb74cd-6239-4260-89f0-36b71fa3ded0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:00.856168 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bb74cd-6239-4260-89f0-36b71fa3ded0 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:00.861951 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb3583-4219-4297-9d13-ebd7824bedb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:00.862560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb3583-4219-4297-9d13-ebd7824bedb2 service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:00.862838 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb3583-4219-4297-9d13-ebd7824bedb2 service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:00.863112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb3583-4219-4297-9d13-ebd7824bedb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:00.865156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b2bb3583-4219-4297-9d13-ebd7824bedb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:00.880319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb3583-4219-4297-9d13-ebd7824bedb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:00.881916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb3583-4219-4297-9d13-ebd7824bedb2 service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:00.894392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1225/2440] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:23:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:23:04.461651 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966837de-b9b7-4de4-b630-608d34cbaed3 service swift] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:04.463074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-966837de-b9b7-4de4-b630-608d34cbaed3 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:04.528916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966837de-b9b7-4de4-b630-608d34cbaed3 service swift] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:04.539717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:04.540374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:04.540657 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:04.540931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] PATH_INFO: `/v3/users/993bf8937c8349e7887b3825a7034367` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:04.542089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] RBAC: Authorizing `identity:delete_user(user_id=993bf8937c8349e7887b3825a7034367)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:04.544078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:04.595808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993bf8937c8349e7887b3825a7034367'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:04.596472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993bf8937c8349e7887b3825a7034367'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:04.604827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993bf8937c8349e7887b3825a7034367'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:04.611980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993bf8937c8349e7887b3825a7034367'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:04.612660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993bf8937c8349e7887b3825a7034367'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:04.632892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1216/2441] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:23:04 2022] DELETE /identity/v3/users/993bf8937c8349e7887b3825a7034367 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:23:05.072627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb3583-4219-4297-9d13-ebd7824bedb2 service placement] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:05.084016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb3583-4219-4297-9d13-ebd7824bedb2 service placement] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:05.149832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:05.150471 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:05.150750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:05.151022 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] PATH_INFO: `/v3/projects/6109b45e7ff14addb54c4bff4fb43df6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:05.152200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6109b45e7ff14addb54c4bff4fb43df6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:05.154167 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:05.203902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6109b45e7ff14addb54c4bff4fb43df6', 'request_id': 'req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.209638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6109b45e7ff14addb54c4bff4fb43df6', 'request_id': 'req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.217097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6109b45e7ff14addb54c4bff4fb43df6', 'request_id': 'req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.229045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] The token cache is being invalidate because project 6109b45e7ff14addb54c4bff4fb43df6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:23:05.229413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.230136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.231831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1226/2442] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:23:05 2022] DELETE /identity/v3/projects/6109b45e7ff14addb54c4bff4fb43df6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:23:05.236859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:05.238386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:05.252453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c592ba96-c3b1-465e-82b0-e7cb0728274e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:05.310703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:05.311412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:05.311714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:05.311984 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] PATH_INFO: `/v3/users/059c8b51434f43f7a3e85581094c9a02` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:05.313427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] RBAC: Authorizing `identity:delete_user(user_id=059c8b51434f43f7a3e85581094c9a02)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:05.315358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:05.321033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:23:05.364867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059c8b51434f43f7a3e85581094c9a02'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.365506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059c8b51434f43f7a3e85581094c9a02'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.373438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059c8b51434f43f7a3e85581094c9a02'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.381635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059c8b51434f43f7a3e85581094c9a02'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.382373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-cebc2e33-5ad6-4167-a3e8-a081c367d318 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059c8b51434f43f7a3e85581094c9a02'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.400172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1217/2443] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:23:05 2022] DELETE /identity/v3/users/059c8b51434f43f7a3e85581094c9a02 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:23:05.581938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:05.583555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:05.608440 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec4dc45-dd6f-47aa-89d5-550f7c53c202 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:05.620319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:23:05.645831 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd20853-de8e-4252-948f-2fde34a708b7 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:05.646319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd20853-de8e-4252-948f-2fde34a708b7 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:05.646420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd20853-de8e-4252-948f-2fde34a708b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:05.669435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-ebd20853-de8e-4252-948f-2fde34a708b7 None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:23:05.670668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ebd20853-de8e-4252-948f-2fde34a708b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:05.673160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:05.673871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:05.674222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:05.674567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] PATH_INFO: `/v3/projects/4777c5b655354f59a160176da328c8ee` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:05.675993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4777c5b655354f59a160176da328c8ee)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:05.677880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:05.732703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1218/2444] 10.222.0.26 () {58 vars in 1009 bytes} [Sat Sep 10 14:23:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:23:05.734129 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4777c5b655354f59a160176da328c8ee', 'request_id': 'req-c01e4f2a-91d6-428f-bacf-97aba0ffa009'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.740624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4777c5b655354f59a160176da328c8ee', 'request_id': 'req-c01e4f2a-91d6-428f-bacf-97aba0ffa009'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.748554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd20853-de8e-4252-948f-2fde34a708b7 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:05.749357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4777c5b655354f59a160176da328c8ee', 'request_id': 'req-c01e4f2a-91d6-428f-bacf-97aba0ffa009'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.749952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ebd20853-de8e-4252-948f-2fde34a708b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:05.761191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] The token cache is being invalidate because project 4777c5b655354f59a160176da328c8ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:23:05.761566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.762294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c01e4f2a-91d6-428f-bacf-97aba0ffa009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:23:05.764075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1227/2445] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:23:05 2022] DELETE /identity/v3/projects/4777c5b655354f59a160176da328c8ee => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:23:05.769478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05c66d-d127-407b-bbb1-06f6b9defae6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:05.769755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05c66d-d127-407b-bbb1-06f6b9defae6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:05.770025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05c66d-d127-407b-bbb1-06f6b9defae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:05.791371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fb05c66d-d127-407b-bbb1-06f6b9defae6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:23:05.792606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fb05c66d-d127-407b-bbb1-06f6b9defae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:05.835431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd20853-de8e-4252-948f-2fde34a708b7 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:05.863068 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1228/2446] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:23:05.871913 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3c9bf-3291-47db-902d-678f51fd87f9 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:05.872058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:23:05.872269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3c9bf-3291-47db-902d-678f51fd87f9 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:05.872458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3c9bf-3291-47db-902d-678f51fd87f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:05.895277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6af3c9bf-3291-47db-902d-678f51fd87f9 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:23:05.896575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6af3c9bf-3291-47db-902d-678f51fd87f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:05.904966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1229/2447] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:23:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:23:05.909998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3c9bf-3291-47db-902d-678f51fd87f9 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:05.914086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352a9c8-d2ed-4d98-afe7-7c9c7e69c86d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:05.914942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352a9c8-d2ed-4d98-afe7-7c9c7e69c86d service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:05.915297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352a9c8-d2ed-4d98-afe7-7c9c7e69c86d service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:05.915696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352a9c8-d2ed-4d98-afe7-7c9c7e69c86d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:05.917673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4352a9c8-d2ed-4d98-afe7-7c9c7e69c86d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:05.924057 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af3c9bf-3291-47db-902d-678f51fd87f9 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:05.932922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1f6d5-40b9-4cd8-b010-31ae7fd8a2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:05.933526 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f6d5-40b9-4cd8-b010-31ae7fd8a2dd admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:05.933808 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f6d5-40b9-4cd8-b010-31ae7fd8a2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:05.934081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce1f6d5-40b9-4cd8-b010-31ae7fd8a2dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:05.934785 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1f6d5-40b9-4cd8-b010-31ae7fd8a2dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:23:05.935090 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1f6d5-40b9-4cd8-b010-31ae7fd8a2dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:05.936733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce1f6d5-40b9-4cd8-b010-31ae7fd8a2dd admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:05.950285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1230/2448] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:23:05.954683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1f6d5-40b9-4cd8-b010-31ae7fd8a2dd admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:05.965672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce1f6d5-40b9-4cd8-b010-31ae7fd8a2dd admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:05.971431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf4f1e-0f4b-414f-88b3-78a6edf4cf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:05.972078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf4f1e-0f4b-414f-88b3-78a6edf4cf97 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:05.972353 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf4f1e-0f4b-414f-88b3-78a6edf4cf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:05.972613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf4f1e-0f4b-414f-88b3-78a6edf4cf97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:05.973261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf4f1e-0f4b-414f-88b3-78a6edf4cf97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:05.975352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf4f1e-0f4b-414f-88b3-78a6edf4cf97 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:05.989528 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352a9c8-d2ed-4d98-afe7-7c9c7e69c86d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:05.991196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352a9c8-d2ed-4d98-afe7-7c9c7e69c86d service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:06.001109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1219/2449] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:23:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 10 14:23:06.008072 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1231/2450] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:23:05 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:23:06.012048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352a9c8-d2ed-4d98-afe7-7c9c7e69c86d service swift] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:06.024089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352a9c8-d2ed-4d98-afe7-7c9c7e69c86d service swift] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:06.130841 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaacaac-1e40-4462-ae45-11b174757f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:06.134815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaacaac-1e40-4462-ae45-11b174757f20 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:06.135373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaacaac-1e40-4462-ae45-11b174757f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:06.137222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaacaac-1e40-4462-ae45-11b174757f20 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:06.138146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaacaac-1e40-4462-ae45-11b174757f20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:06.142808 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaacaac-1e40-4462-ae45-11b174757f20 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:06.187085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1220/2451] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:23:06 2022] POST /identity/v3/users => generated 287 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:23:06.187481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf4f1e-0f4b-414f-88b3-78a6edf4cf97 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:06.194236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:23:06.201308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf4f1e-0f4b-414f-88b3-78a6edf4cf97 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:06.217222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b8ed1-1542-4384-9ead-21f9d556073f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:06.218570 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b8ed1-1542-4384-9ead-21f9d556073f admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:06.219896 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b8ed1-1542-4384-9ead-21f9d556073f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:06.220324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911b8ed1-1542-4384-9ead-21f9d556073f admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:06.221286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b8ed1-1542-4384-9ead-21f9d556073f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:06.223496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911b8ed1-1542-4384-9ead-21f9d556073f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:06.236333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1232/2452] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:23:06.240471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaacaac-1e40-4462-ae45-11b174757f20 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:06.267774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaacaac-1e40-4462-ae45-11b174757f20 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:06.273524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2abe77a-34a8-48bd-842d-1351b50b6566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:06.274500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe77a-34a8-48bd-842d-1351b50b6566 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:06.274942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe77a-34a8-48bd-842d-1351b50b6566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:06.275388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2abe77a-34a8-48bd-842d-1351b50b6566 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:06.276294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abe77a-34a8-48bd-842d-1351b50b6566 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:06.278555 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2abe77a-34a8-48bd-842d-1351b50b6566 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:06.296246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1221/2453] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:23:06 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:23:06.297829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b8ed1-1542-4384-9ead-21f9d556073f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:06.317214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911b8ed1-1542-4384-9ead-21f9d556073f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:06.323275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6dd2f-f865-4028-982b-aba6c220afc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:06.324238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6dd2f-f865-4028-982b-aba6c220afc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:06.324612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6dd2f-f865-4028-982b-aba6c220afc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:06.325033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de6dd2f-f865-4028-982b-aba6c220afc4 admin admin] PATH_INFO: `/v3/projects/b4ebb6e9554f4c0d985bf90411a4e36f/users/b13d21124dbf41459543426847b67bea/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:06.352037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6dd2f-f865-4028-982b-aba6c220afc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4ebb6e9554f4c0d985bf90411a4e36f, user_id=b13d21124dbf41459543426847b67bea, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:06.355015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de6dd2f-f865-4028-982b-aba6c220afc4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:06.373586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1233/2454] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:23:06 2022] PUT /identity/v3/projects/b4ebb6e9554f4c0d985bf90411a4e36f/users/b13d21124dbf41459543426847b67bea/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:23:06.379837 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013883a-243f-4620-b024-8f77e3005d16 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:06.380131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013883a-243f-4620-b024-8f77e3005d16 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:06.380399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013883a-243f-4620-b024-8f77e3005d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:06.404799 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c013883a-243f-4620-b024-8f77e3005d16 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:23:06.406931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c013883a-243f-4620-b024-8f77e3005d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:06.465464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:23:06.482915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1222/2455] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:23:06.491535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6dd2f-f865-4028-982b-aba6c220afc4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:06.493935 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0de6dd2f-f865-4028-982b-aba6c220afc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:06.559913 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de6dd2f-f865-4028-982b-aba6c220afc4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:06.564516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0793aa7b-742a-431f-87a1-9f51ce53ef40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:06.565327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793aa7b-742a-431f-87a1-9f51ce53ef40 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:06.565691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793aa7b-742a-431f-87a1-9f51ce53ef40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:06.566031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0793aa7b-742a-431f-87a1-9f51ce53ef40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:06.575730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793aa7b-742a-431f-87a1-9f51ce53ef40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:06.577191 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0793aa7b-742a-431f-87a1-9f51ce53ef40 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:06.587825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1234/2456] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:23:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:23:10.321049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a15e7-9521-4f08-b1ea-66f27eebb458 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:10.321597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a15e7-9521-4f08-b1ea-66f27eebb458 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:10.321597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a15e7-9521-4f08-b1ea-66f27eebb458 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:10.352208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-4d3a15e7-9521-4f08-b1ea-66f27eebb458 None None] MFA Rules not processed for user `f4638a1936f5467db502bda82320b113`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:23:10.353484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4d3a15e7-9521-4f08-b1ea-66f27eebb458 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:10.498568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1223/2457] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:23:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:23:12.859761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1417d-a5ed-4156-9530-f10d13c9e48d None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:12.860523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1417d-a5ed-4156-9530-f10d13c9e48d None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:12.860523 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c1417d-a5ed-4156-9530-f10d13c9e48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:12.902224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f7c1417d-a5ed-4156-9530-f10d13c9e48d None None] MFA Rules not processed for user `b13d21124dbf41459543426847b67bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:23:12.904309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f7c1417d-a5ed-4156-9530-f10d13c9e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:13.012070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1235/2458] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:23:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:23:13.016740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a15e7-9521-4f08-b1ea-66f27eebb458 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:13.040386 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a15e7-9521-4f08-b1ea-66f27eebb458 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:13.064325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155e48a-d224-42c2-ada9-57a2f80a2744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:13.064970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e48a-d224-42c2-ada9-57a2f80a2744 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:13.065250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e48a-d224-42c2-ada9-57a2f80a2744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:13.065524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155e48a-d224-42c2-ada9-57a2f80a2744 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:13.066193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155e48a-d224-42c2-ada9-57a2f80a2744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:13.068159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155e48a-d224-42c2-ada9-57a2f80a2744 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:13.165741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1224/2459] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:23:13 2022] POST /identity/v3/projects => generated 373 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:23:13.165967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1417d-a5ed-4156-9530-f10d13c9e48d None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:13.181182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c1417d-a5ed-4156-9530-f10d13c9e48d None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:13.280311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea888aa-6463-4990-badc-ecdbdabdd008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:13.280925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea888aa-6463-4990-badc-ecdbdabdd008 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:13.281954 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea888aa-6463-4990-badc-ecdbdabdd008 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:13.282112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea888aa-6463-4990-badc-ecdbdabdd008 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:13.282840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea888aa-6463-4990-badc-ecdbdabdd008 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:13.284834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea888aa-6463-4990-badc-ecdbdabdd008 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:13.292772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:23:13.308081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1236/2460] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:23:13 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:23:13.311913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155e48a-d224-42c2-ada9-57a2f80a2744 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:13.322547 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155e48a-d224-42c2-ada9-57a2f80a2744 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:13.327823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7b7eb-6ad2-4e8c-911a-cb8e74046c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:13.329021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7b7eb-6ad2-4e8c-911a-cb8e74046c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:13.329298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7b7eb-6ad2-4e8c-911a-cb8e74046c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:13.329617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7b7eb-6ad2-4e8c-911a-cb8e74046c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:13.330405 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7b7eb-6ad2-4e8c-911a-cb8e74046c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:13.332241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7b7eb-6ad2-4e8c-911a-cb8e74046c4f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:13.344477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1225/2461] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:23:13.348777 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea888aa-6463-4990-badc-ecdbdabdd008 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:13.358930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea888aa-6463-4990-badc-ecdbdabdd008 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:13.364760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0313a-2a70-4994-8a04-0f4a862adca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:13.365547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0313a-2a70-4994-8a04-0f4a862adca7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:13.365764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0313a-2a70-4994-8a04-0f4a862adca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:13.365945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee0313a-2a70-4994-8a04-0f4a862adca7 admin admin] PATH_INFO: `/v3/projects/f53c623bc71a487cad8d53dc5c66b349/users/4e94dd8f024c4e6487f89481c614eb27/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:13.382669 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0313a-2a70-4994-8a04-0f4a862adca7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f53c623bc71a487cad8d53dc5c66b349, user_id=4e94dd8f024c4e6487f89481c614eb27, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:13.385703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee0313a-2a70-4994-8a04-0f4a862adca7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:13.401571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1237/2462] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:23:13 2022] PUT /identity/v3/projects/f53c623bc71a487cad8d53dc5c66b349/users/4e94dd8f024c4e6487f89481c614eb27/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:23:13.405645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7b7eb-6ad2-4e8c-911a-cb8e74046c4f admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:13.416941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7b7eb-6ad2-4e8c-911a-cb8e74046c4f admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:13.481675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef31df-f553-4db3-8fa7-9de543b1b697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:13.482282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef31df-f553-4db3-8fa7-9de543b1b697 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:13.482553 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef31df-f553-4db3-8fa7-9de543b1b697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:13.482816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef31df-f553-4db3-8fa7-9de543b1b697 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:13.483487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef31df-f553-4db3-8fa7-9de543b1b697 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:13.486079 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef31df-f553-4db3-8fa7-9de543b1b697 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:13.496262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1226/2463] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:23:13.500688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0313a-2a70-4994-8a04-0f4a862adca7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:13.511109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee0313a-2a70-4994-8a04-0f4a862adca7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:13.516456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd4112-916c-4665-8768-086bfa078170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:13.517210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd4112-916c-4665-8768-086bfa078170 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:13.517488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd4112-916c-4665-8768-086bfa078170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:13.517764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddd4112-916c-4665-8768-086bfa078170 admin admin] PATH_INFO: `/v3/projects/f53c623bc71a487cad8d53dc5c66b349/users/4e94dd8f024c4e6487f89481c614eb27/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:13.519705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd4112-916c-4665-8768-086bfa078170 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f53c623bc71a487cad8d53dc5c66b349, user_id=4e94dd8f024c4e6487f89481c614eb27, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:13.522364 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddd4112-916c-4665-8768-086bfa078170 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:13.533550 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1238/2464] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:23:13 2022] PUT /identity/v3/projects/f53c623bc71a487cad8d53dc5c66b349/users/4e94dd8f024c4e6487f89481c614eb27/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:23:18.988268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49652e-e058-4b8b-adcf-8e867b00fe78 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:18.988864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49652e-e058-4b8b-adcf-8e867b00fe78 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:18.989232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49652e-e058-4b8b-adcf-8e867b00fe78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:19.024740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-0e49652e-e058-4b8b-adcf-8e867b00fe78 None None] MFA Rules not processed for user `4e94dd8f024c4e6487f89481c614eb27`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:23:19.025500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0e49652e-e058-4b8b-adcf-8e867b00fe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:19.082817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:23:19.116409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1227/2465] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:23:18 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:23:19.124303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd4112-916c-4665-8768-086bfa078170 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:19.127135 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4ddd4112-916c-4665-8768-086bfa078170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:19.245194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddd4112-916c-4665-8768-086bfa078170 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:19.251546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0a6043-96e5-44fa-8cf0-9068d4fb7194 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:19.253360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a6043-96e5-44fa-8cf0-9068d4fb7194 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:19.253360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a6043-96e5-44fa-8cf0-9068d4fb7194 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:19.253360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0a6043-96e5-44fa-8cf0-9068d4fb7194 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:19.321261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a6043-96e5-44fa-8cf0-9068d4fb7194 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:19.322893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0a6043-96e5-44fa-8cf0-9068d4fb7194 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:19.337482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1239/2466] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:23:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:23:20.425924 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1145146-75a9-48c3-91ed-4c9904a22bff None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:20.426439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1145146-75a9-48c3-91ed-4c9904a22bff None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:20.426439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1145146-75a9-48c3-91ed-4c9904a22bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:20.445403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a1145146-75a9-48c3-91ed-4c9904a22bff None None] MFA Rules not processed for user `b13d21124dbf41459543426847b67bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:23:20.446600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a1145146-75a9-48c3-91ed-4c9904a22bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:20.453461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1228/2467] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:23:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:23:29.240962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960ac45-42e1-4d26-ae8f-e1333bb316bb None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:29.241442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960ac45-42e1-4d26-ae8f-e1333bb316bb None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:29.241847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960ac45-42e1-4d26-ae8f-e1333bb316bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:29.262205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f960ac45-42e1-4d26-ae8f-e1333bb316bb None None] MFA Rules not processed for user `d75640e511e64661826c8b8d7aac2ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:23:29.263597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f960ac45-42e1-4d26-ae8f-e1333bb316bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:29.314713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:23:29.327054 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1240/2468] 10.222.0.26 () {58 vars in 967 bytes} [Sat Sep 10 14:23:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:23:29.339958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1145146-75a9-48c3-91ed-4c9904a22bff None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:29.341310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a1145146-75a9-48c3-91ed-4c9904a22bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:29.411931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1145146-75a9-48c3-91ed-4c9904a22bff None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:29.417153 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869676d-e030-4f5e-80e3-21c0de4433c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc0a7e1b772b4703ac2e5ec33b5ecf6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:29.417653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869676d-e030-4f5e-80e3-21c0de4433c6 service swift] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:29.417931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869676d-e030-4f5e-80e3-21c0de4433c6 service swift] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:29.418253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2869676d-e030-4f5e-80e3-21c0de4433c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:29.427471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869676d-e030-4f5e-80e3-21c0de4433c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:29.429097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2869676d-e030-4f5e-80e3-21c0de4433c6 service swift] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:29.447782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1229/2469] 10.222.0.26 () {60 vars in 1384 bytes} [Sat Sep 10 14:23:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 10 14:23:31.900603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960ac45-42e1-4d26-ae8f-e1333bb316bb None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:31.958342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960ac45-42e1-4d26-ae8f-e1333bb316bb None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:31.961858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1d158-bc92-4afc-befd-d5b591a909ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:31.962457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1d158-bc92-4afc-befd-d5b591a909ad service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:31.962736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1d158-bc92-4afc-befd-d5b591a909ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:31.963002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1d158-bc92-4afc-befd-d5b591a909ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:31.964834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-85d1d158-bc92-4afc-befd-d5b591a909ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:31.979325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1d158-bc92-4afc-befd-d5b591a909ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:31.980804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d1d158-bc92-4afc-befd-d5b591a909ad service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:31.992464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1241/2470] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:23:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:23:33.178197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869676d-e030-4f5e-80e3-21c0de4433c6 service swift] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:33.179818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2869676d-e030-4f5e-80e3-21c0de4433c6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:33.192022 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2869676d-e030-4f5e-80e3-21c0de4433c6 service swift] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:33.195512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b48c0-968f-49fb-b0bf-f802ceebb6da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:33.196097 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b48c0-968f-49fb-b0bf-f802ceebb6da service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:33.196367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b48c0-968f-49fb-b0bf-f802ceebb6da service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:33.196631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b48c0-968f-49fb-b0bf-f802ceebb6da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:33.198287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2d9b48c0-968f-49fb-b0bf-f802ceebb6da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:33.211921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b48c0-968f-49fb-b0bf-f802ceebb6da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:33.213328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b48c0-968f-49fb-b0bf-f802ceebb6da service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:33.224867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1230/2471] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:23:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:23:41.776860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1d158-bc92-4afc-befd-d5b591a909ad service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:23:41.785608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d1d158-bc92-4afc-befd-d5b591a909ad service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:23:41.789196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3f78a-3692-4129-860a-aba77469a9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:23:41.789784 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3f78a-3692-4129-860a-aba77469a9fa service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:23:41.790055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3f78a-3692-4129-860a-aba77469a9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:23:41.790321 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e3f78a-3692-4129-860a-aba77469a9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:23:41.792055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-95e3f78a-3692-4129-860a-aba77469a9fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:23:41.853719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3f78a-3692-4129-860a-aba77469a9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:23:41.855084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e3f78a-3692-4129-860a-aba77469a9fa service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:23:41.865359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1242/2472] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:23:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:23:43.219269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=47 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:23:51.860330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=59 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:04.571472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b48c0-968f-49fb-b0bf-f802ceebb6da service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:04.580046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b48c0-968f-49fb-b0bf-f802ceebb6da service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:04.583822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e02a-4b88-403c-a410-8044f7e80a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:04.584422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e02a-4b88-403c-a410-8044f7e80a35 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:04.584749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e02a-4b88-403c-a410-8044f7e80a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:04.585028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e597e02a-4b88-403c-a410-8044f7e80a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:04.586780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e597e02a-4b88-403c-a410-8044f7e80a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:04.649957 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597e02a-4b88-403c-a410-8044f7e80a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:04.651388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e597e02a-4b88-403c-a410-8044f7e80a35 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:04.661633 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1231/2473] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:05.509070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3f78a-3692-4129-860a-aba77469a9fa service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:05.518139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e3f78a-3692-4129-860a-aba77469a9fa service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:05.522251 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec84043-e4ad-406d-9900-af4f04f6964d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:05.522863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec84043-e4ad-406d-9900-af4f04f6964d service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:05.523137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec84043-e4ad-406d-9900-af4f04f6964d service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:05.523822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec84043-e4ad-406d-9900-af4f04f6964d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:05.534298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec84043-e4ad-406d-9900-af4f04f6964d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:05.535373 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec84043-e4ad-406d-9900-af4f04f6964d service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:05.545221 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1243/2474] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:08.872085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e02a-4b88-403c-a410-8044f7e80a35 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:08.873567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e597e02a-4b88-403c-a410-8044f7e80a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:08.890277 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e597e02a-4b88-403c-a410-8044f7e80a35 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:08.896639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:08.897316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:08.897907 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:08.898446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] PATH_INFO: `/v3/users/8ea8ba61dfa54cbc9a45543d20f31e76` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:08.916997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea8ba61dfa54cbc9a45543d20f31e76)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:08.919242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:08.975632 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea8ba61dfa54cbc9a45543d20f31e76'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:08.976293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea8ba61dfa54cbc9a45543d20f31e76'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:08.987825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea8ba61dfa54cbc9a45543d20f31e76'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:08.998192 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea8ba61dfa54cbc9a45543d20f31e76'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:08.999007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bc72eb0a-92b1-4aee-b264-f3bce90ac07a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea8ba61dfa54cbc9a45543d20f31e76'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:09.025403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1232/2475] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:08 2022] DELETE /identity/v3/users/8ea8ba61dfa54cbc9a45543d20f31e76 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:09.214232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec84043-e4ad-406d-9900-af4f04f6964d service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:09.228892 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec84043-e4ad-406d-9900-af4f04f6964d service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:09.298361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:09.299243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:09.299542 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:09.299812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] PATH_INFO: `/v3/projects/6efda1ce3d4d4dbbb5e8d5005902924d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:09.310804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6efda1ce3d4d4dbbb5e8d5005902924d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:09.312863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:09.359821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6efda1ce3d4d4dbbb5e8d5005902924d', 'request_id': 'req-b1107bef-3e9e-4173-b2a0-241bd8112a7e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:09.365897 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6efda1ce3d4d4dbbb5e8d5005902924d', 'request_id': 'req-b1107bef-3e9e-4173-b2a0-241bd8112a7e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:09.373626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6efda1ce3d4d4dbbb5e8d5005902924d', 'request_id': 'req-b1107bef-3e9e-4173-b2a0-241bd8112a7e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:09.385795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] The token cache is being invalidate because project 6efda1ce3d4d4dbbb5e8d5005902924d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:09.386071 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:09.386749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-b1107bef-3e9e-4173-b2a0-241bd8112a7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:09.388485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1244/2476] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:09 2022] DELETE /identity/v3/projects/6efda1ce3d4d4dbbb5e8d5005902924d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:09.394151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784d983-9d12-4d6a-8b8c-f2304211576b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:09.394442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784d983-9d12-4d6a-8b8c-f2304211576b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:09.394768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784d983-9d12-4d6a-8b8c-f2304211576b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:09.415006 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-6784d983-9d12-4d6a-8b8c-f2304211576b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:09.416239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6784d983-9d12-4d6a-8b8c-f2304211576b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:09.468823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:09.481756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1233/2477] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:24:09.491626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4606-cf21-4d85-8d82-328ef0caaf16 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:09.491905 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4606-cf21-4d85-8d82-328ef0caaf16 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:09.492175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c4606-cf21-4d85-8d82-328ef0caaf16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:09.511852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ad0c4606-cf21-4d85-8d82-328ef0caaf16 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:09.513030 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ad0c4606-cf21-4d85-8d82-328ef0caaf16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:09.521083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1245/2478] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:09.525177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784d983-9d12-4d6a-8b8c-f2304211576b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:09.535195 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784d983-9d12-4d6a-8b8c-f2304211576b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:09.539748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb872c15-b160-483d-b861-e81e4cfcc997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:09.540348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb872c15-b160-483d-b861-e81e4cfcc997 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:09.540623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb872c15-b160-483d-b861-e81e4cfcc997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:09.540894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb872c15-b160-483d-b861-e81e4cfcc997 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:09.541539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb872c15-b160-483d-b861-e81e4cfcc997 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:24:09.541838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb872c15-b160-483d-b861-e81e4cfcc997 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:09.543463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb872c15-b160-483d-b861-e81e4cfcc997 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:09.555418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1234/2479] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:24:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:24:09.559660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4606-cf21-4d85-8d82-328ef0caaf16 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:09.569667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c4606-cf21-4d85-8d82-328ef0caaf16 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:09.574186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82938a-0c87-4767-aa7d-20554cccfbf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:09.574763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82938a-0c87-4767-aa7d-20554cccfbf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:09.575034 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82938a-0c87-4767-aa7d-20554cccfbf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:09.575299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a82938a-0c87-4767-aa7d-20554cccfbf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:09.575946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82938a-0c87-4767-aa7d-20554cccfbf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:09.577840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a82938a-0c87-4767-aa7d-20554cccfbf9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:09.614556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1246/2480] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:09 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:09.618730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb872c15-b160-483d-b861-e81e4cfcc997 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:09.629574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb872c15-b160-483d-b861-e81e4cfcc997 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:09.685397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6c96b-e78f-4e5d-a8a5-86d761361c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:09.685996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c96b-e78f-4e5d-a8a5-86d761361c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:09.686277 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c96b-e78f-4e5d-a8a5-86d761361c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:09.686551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6c96b-e78f-4e5d-a8a5-86d761361c75 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:09.687170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6c96b-e78f-4e5d-a8a5-86d761361c75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:09.689843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6c96b-e78f-4e5d-a8a5-86d761361c75 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:09.710805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1235/2481] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:09 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:09.715122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82938a-0c87-4767-aa7d-20554cccfbf9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:09.725369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a82938a-0c87-4767-aa7d-20554cccfbf9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:09.729954 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fea2fd-aab1-499b-8857-525643963fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:09.730537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fea2fd-aab1-499b-8857-525643963fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:09.730815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fea2fd-aab1-499b-8857-525643963fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:09.731088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fea2fd-aab1-499b-8857-525643963fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:09.731763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fea2fd-aab1-499b-8857-525643963fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:09.733370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fea2fd-aab1-499b-8857-525643963fa0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:09.743936 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1247/2482] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:09.747726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6c96b-e78f-4e5d-a8a5-86d761361c75 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:09.758251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6c96b-e78f-4e5d-a8a5-86d761361c75 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:09.762901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bab758d-2a71-430e-bba0-e5e380c4085e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:09.763521 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab758d-2a71-430e-bba0-e5e380c4085e admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:09.763803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab758d-2a71-430e-bba0-e5e380c4085e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:09.764076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bab758d-2a71-430e-bba0-e5e380c4085e admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:09.764784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab758d-2a71-430e-bba0-e5e380c4085e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:09.766527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bab758d-2a71-430e-bba0-e5e380c4085e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:09.776934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1236/2483] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:09.780987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fea2fd-aab1-499b-8857-525643963fa0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:09.791646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fea2fd-aab1-499b-8857-525643963fa0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:09.796227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4752c0-c77a-4897-8ba8-513babc01ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:09.796860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4752c0-c77a-4897-8ba8-513babc01ea2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:09.797135 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4752c0-c77a-4897-8ba8-513babc01ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:09.797404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4752c0-c77a-4897-8ba8-513babc01ea2 admin admin] PATH_INFO: `/v3/projects/c00eacefc2f4439699776b3cf4a439a6/users/8c1499a50b984de4a0293767f8327b2c/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:09.812344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4752c0-c77a-4897-8ba8-513babc01ea2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c00eacefc2f4439699776b3cf4a439a6, user_id=8c1499a50b984de4a0293767f8327b2c, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:09.814999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4752c0-c77a-4897-8ba8-513babc01ea2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:09.818055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:09.830379 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1248/2484] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:09 2022] PUT /identity/v3/projects/c00eacefc2f4439699776b3cf4a439a6/users/8c1499a50b984de4a0293767f8327b2c/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:09.840741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e1fbd-b292-4392-9c52-7abbba4b04a0 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:09.841026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e1fbd-b292-4392-9c52-7abbba4b04a0 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:09.841321 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e1fbd-b292-4392-9c52-7abbba4b04a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:09.869935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-b56e1fbd-b292-4392-9c52-7abbba4b04a0 None None] MFA Rules not processed for user `8c1499a50b984de4a0293767f8327b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:09.871142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b56e1fbd-b292-4392-9c52-7abbba4b04a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:09.937126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1237/2485] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:09.948902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4752c0-c77a-4897-8ba8-513babc01ea2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:09.950584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3b4752c0-c77a-4897-8ba8-513babc01ea2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:10.024325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4752c0-c77a-4897-8ba8-513babc01ea2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:10.029685 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb79b9-2729-4281-b53e-10fc74e6f873 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:10.030278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb79b9-2729-4281-b53e-10fc74e6f873 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:10.030554 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb79b9-2729-4281-b53e-10fc74e6f873 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:10.030829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb79b9-2729-4281-b53e-10fc74e6f873 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:10.038572 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb79b9-2729-4281-b53e-10fc74e6f873 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:10.040260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb79b9-2729-4281-b53e-10fc74e6f873 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:10.048999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1249/2486] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:12.722090 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e1fbd-b292-4392-9c52-7abbba4b04a0 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:12.774065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:12.787784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e1fbd-b292-4392-9c52-7abbba4b04a0 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:12.792554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:12.793177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:12.793447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:12.793716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] PATH_INFO: `/v3/users/8c1499a50b984de4a0293767f8327b2c` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:12.794835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c1499a50b984de4a0293767f8327b2c)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:12.796797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:12.844314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c1499a50b984de4a0293767f8327b2c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:12.845889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c1499a50b984de4a0293767f8327b2c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:12.854818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c1499a50b984de4a0293767f8327b2c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:12.862812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c1499a50b984de4a0293767f8327b2c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:12.863550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c1499a50b984de4a0293767f8327b2c'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:12.888335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1238/2487] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:12 2022] DELETE /identity/v3/users/8c1499a50b984de4a0293767f8327b2c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:24:12.895166 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54879fb8-b703-4e84-927e-2bacef548c26 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:12.895705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54879fb8-b703-4e84-927e-2bacef548c26 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:12.895993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54879fb8-b703-4e84-927e-2bacef548c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:12.925321 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-54879fb8-b703-4e84-927e-2bacef548c26 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:12.927009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-54879fb8-b703-4e84-927e-2bacef548c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:13.015583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1250/2488] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:13.025578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:13.034739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:13.094300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464fc53-916d-46a4-b0d6-0f3b07a6dd02 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:13.098566 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701f7f85-04e9-48e0-92be-426513897476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:13.099191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f7f85-04e9-48e0-92be-426513897476 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.099499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f7f85-04e9-48e0-92be-426513897476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.099773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701f7f85-04e9-48e0-92be-426513897476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.108727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f7f85-04e9-48e0-92be-426513897476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:13.110821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701f7f85-04e9-48e0-92be-426513897476 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:13.120628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1239/2489] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:13.399111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54879fb8-b703-4e84-927e-2bacef548c26 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:13.400618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-54879fb8-b703-4e84-927e-2bacef548c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:13.413004 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54879fb8-b703-4e84-927e-2bacef548c26 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:13.417597 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:13.418320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.418681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.419008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] PATH_INFO: `/v3/projects/c00eacefc2f4439699776b3cf4a439a6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.420106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c00eacefc2f4439699776b3cf4a439a6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:13.422048 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:13.470618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c00eacefc2f4439699776b3cf4a439a6', 'request_id': 'req-4a659f95-5a9b-4836-aaa0-5074720f283a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:13.475952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c00eacefc2f4439699776b3cf4a439a6', 'request_id': 'req-4a659f95-5a9b-4836-aaa0-5074720f283a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:13.483310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c00eacefc2f4439699776b3cf4a439a6', 'request_id': 'req-4a659f95-5a9b-4836-aaa0-5074720f283a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:13.494666 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] The token cache is being invalidate because project c00eacefc2f4439699776b3cf4a439a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:13.494945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:13.495599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-4a659f95-5a9b-4836-aaa0-5074720f283a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:13.497055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1251/2490] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:13 2022] DELETE /identity/v3/projects/c00eacefc2f4439699776b3cf4a439a6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:13.502184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b00221-b984-4a0d-aa64-97238bdfdf56 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.502461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b00221-b984-4a0d-aa64-97238bdfdf56 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.502726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b00221-b984-4a0d-aa64-97238bdfdf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.522429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-56b00221-b984-4a0d-aa64-97238bdfdf56 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:13.523644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-56b00221-b984-4a0d-aa64-97238bdfdf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:13.583377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1240/2491] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:13.591065 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b4207-cb69-44d9-8a38-085fd14c8f0f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.591370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b4207-cb69-44d9-8a38-085fd14c8f0f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.591664 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b4207-cb69-44d9-8a38-085fd14c8f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.600442 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:13.613181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-737b4207-cb69-44d9-8a38-085fd14c8f0f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:13.614425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-737b4207-cb69-44d9-8a38-085fd14c8f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:13.622356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1252/2492] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:13.627864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b00221-b984-4a0d-aa64-97238bdfdf56 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:13.639028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b00221-b984-4a0d-aa64-97238bdfdf56 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:13.644687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e5a6f-16ec-49a6-9cfe-86019ac4b234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:13.645280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e5a6f-16ec-49a6-9cfe-86019ac4b234 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.645618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e5a6f-16ec-49a6-9cfe-86019ac4b234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.645886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06e5a6f-16ec-49a6-9cfe-86019ac4b234 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.646517 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e5a6f-16ec-49a6-9cfe-86019ac4b234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:24:13.646814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e5a6f-16ec-49a6-9cfe-86019ac4b234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:13.648424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06e5a6f-16ec-49a6-9cfe-86019ac4b234 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:13.656712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:13.660815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1241/2493] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:24:13.665188 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b4207-cb69-44d9-8a38-085fd14c8f0f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:13.674420 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b4207-cb69-44d9-8a38-085fd14c8f0f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:13.679018 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a536ccd-6039-46da-a88a-3cbf8251dc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:13.679649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536ccd-6039-46da-a88a-3cbf8251dc28 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.679920 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536ccd-6039-46da-a88a-3cbf8251dc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.680186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a536ccd-6039-46da-a88a-3cbf8251dc28 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.680800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a536ccd-6039-46da-a88a-3cbf8251dc28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:13.682690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a536ccd-6039-46da-a88a-3cbf8251dc28 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:13.710012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1253/2494] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:13 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:24:13.714139 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e5a6f-16ec-49a6-9cfe-86019ac4b234 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:13.724463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06e5a6f-16ec-49a6-9cfe-86019ac4b234 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:13.780221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2acd8f-d169-4b05-9be7-ac3496f7c7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:13.780816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2acd8f-d169-4b05-9be7-ac3496f7c7dc admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.781087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2acd8f-d169-4b05-9be7-ac3496f7c7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.781352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2acd8f-d169-4b05-9be7-ac3496f7c7dc admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.782037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2acd8f-d169-4b05-9be7-ac3496f7c7dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:13.784409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2acd8f-d169-4b05-9be7-ac3496f7c7dc admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:13.805259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1242/2495] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:13 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:13.809333 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a536ccd-6039-46da-a88a-3cbf8251dc28 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:13.820043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a536ccd-6039-46da-a88a-3cbf8251dc28 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:13.824584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dbd7a-67a7-45f0-84fa-fb49632da951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:13.825176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbd7a-67a7-45f0-84fa-fb49632da951 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.825448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbd7a-67a7-45f0-84fa-fb49632da951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.825712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dbd7a-67a7-45f0-84fa-fb49632da951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.826346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dbd7a-67a7-45f0-84fa-fb49632da951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:13.827974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dbd7a-67a7-45f0-84fa-fb49632da951 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:13.838452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1254/2496] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:13.842241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2acd8f-d169-4b05-9be7-ac3496f7c7dc admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:13.852440 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2acd8f-d169-4b05-9be7-ac3496f7c7dc admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:13.856919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3312080-8c04-47cb-bba9-50bdf114d9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:13.857504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3312080-8c04-47cb-bba9-50bdf114d9e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.857776 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3312080-8c04-47cb-bba9-50bdf114d9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.858042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3312080-8c04-47cb-bba9-50bdf114d9e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.858668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3312080-8c04-47cb-bba9-50bdf114d9e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:13.860582 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3312080-8c04-47cb-bba9-50bdf114d9e5 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:13.871477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1243/2497] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:13.875583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dbd7a-67a7-45f0-84fa-fb49632da951 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:13.886592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dbd7a-67a7-45f0-84fa-fb49632da951 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:13.891420 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d24f4f-1e8c-4fe5-b907-f9dc28430205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:13.892249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d24f4f-1e8c-4fe5-b907-f9dc28430205 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.892530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d24f4f-1e8c-4fe5-b907-f9dc28430205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.892803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d24f4f-1e8c-4fe5-b907-f9dc28430205 admin admin] PATH_INFO: `/v3/projects/8bacbd02c499422d9d8982c361550de1/users/c83bcebeab5e484690bbcb29a97f165d/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.907664 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d24f4f-1e8c-4fe5-b907-f9dc28430205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bacbd02c499422d9d8982c361550de1, user_id=c83bcebeab5e484690bbcb29a97f165d, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:13.910358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d24f4f-1e8c-4fe5-b907-f9dc28430205 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:13.924992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1255/2498] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:13 2022] PUT /identity/v3/projects/8bacbd02c499422d9d8982c361550de1/users/c83bcebeab5e484690bbcb29a97f165d/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:13.936340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bb2aa-9b7a-48d7-8f50-5890fa30e6d3 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:13.936649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bb2aa-9b7a-48d7-8f50-5890fa30e6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:13.936917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bb2aa-9b7a-48d7-8f50-5890fa30e6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:13.964389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-f32bb2aa-9b7a-48d7-8f50-5890fa30e6d3 None None] MFA Rules not processed for user `c83bcebeab5e484690bbcb29a97f165d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:13.965584 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f32bb2aa-9b7a-48d7-8f50-5890fa30e6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:14.030166 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1244/2499] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:14.035365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d24f4f-1e8c-4fe5-b907-f9dc28430205 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.098181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d24f4f-1e8c-4fe5-b907-f9dc28430205 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.103378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b38bc-1eb6-42be-b3ec-c7515b2b5bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.104025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b38bc-1eb6-42be-b3ec-c7515b2b5bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.104302 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b38bc-1eb6-42be-b3ec-c7515b2b5bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.104575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b38bc-1eb6-42be-b3ec-c7515b2b5bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.105358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b38bc-1eb6-42be-b3ec-c7515b2b5bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.107409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b38bc-1eb6-42be-b3ec-c7515b2b5bbe admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.136240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1256/2500] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:14 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:14.140637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bb2aa-9b7a-48d7-8f50-5890fa30e6d3 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.151781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bb2aa-9b7a-48d7-8f50-5890fa30e6d3 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.211649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b362c-62aa-4178-8c8f-82cee20e17ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.212249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b362c-62aa-4178-8c8f-82cee20e17ff admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.212530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b362c-62aa-4178-8c8f-82cee20e17ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.212809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79b362c-62aa-4178-8c8f-82cee20e17ff admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.213466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b362c-62aa-4178-8c8f-82cee20e17ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.215404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79b362c-62aa-4178-8c8f-82cee20e17ff admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.230033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:14.238290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1245/2501] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:14 2022] POST /identity/v3/users => generated 294 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:24:14.242496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b38bc-1eb6-42be-b3ec-c7515b2b5bbe admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.255150 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b38bc-1eb6-42be-b3ec-c7515b2b5bbe admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.259820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135a439-d9c5-43b8-966e-6196d183265e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.260399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a439-d9c5-43b8-966e-6196d183265e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.260674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a439-d9c5-43b8-966e-6196d183265e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.260939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8135a439-d9c5-43b8-966e-6196d183265e admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.262069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135a439-d9c5-43b8-966e-6196d183265e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.263932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8135a439-d9c5-43b8-966e-6196d183265e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.276312 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1257/2502] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:14.280509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b362c-62aa-4178-8c8f-82cee20e17ff admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.292170 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79b362c-62aa-4178-8c8f-82cee20e17ff admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.296848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691392a5-1109-442c-a5ff-31d18fe87420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.297473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691392a5-1109-442c-a5ff-31d18fe87420 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.297895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691392a5-1109-442c-a5ff-31d18fe87420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.298175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691392a5-1109-442c-a5ff-31d18fe87420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.298848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691392a5-1109-442c-a5ff-31d18fe87420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.300491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691392a5-1109-442c-a5ff-31d18fe87420 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.310852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1246/2503] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:14.315623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135a439-d9c5-43b8-966e-6196d183265e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.326189 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8135a439-d9c5-43b8-966e-6196d183265e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.331676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54186b25-c9c4-4f3e-bf3d-f6cc65af0024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.332350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54186b25-c9c4-4f3e-bf3d-f6cc65af0024 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.332629 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54186b25-c9c4-4f3e-bf3d-f6cc65af0024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.332904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54186b25-c9c4-4f3e-bf3d-f6cc65af0024 admin admin] PATH_INFO: `/v3/projects/e9f458c36b9b41a9af3e0bc920e7d550/users/8cbb2b875006401cb6276d606d49c5fa/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.349415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54186b25-c9c4-4f3e-bf3d-f6cc65af0024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f458c36b9b41a9af3e0bc920e7d550, user_id=8cbb2b875006401cb6276d606d49c5fa, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.352154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54186b25-c9c4-4f3e-bf3d-f6cc65af0024 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.367487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1258/2504] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:14 2022] PUT /identity/v3/projects/e9f458c36b9b41a9af3e0bc920e7d550/users/8cbb2b875006401cb6276d606d49c5fa/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:24:14.379018 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ea03e-b684-4059-823f-b67c7d1cf46d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.379332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ea03e-b684-4059-823f-b67c7d1cf46d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.379674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ea03e-b684-4059-823f-b67c7d1cf46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.409880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-783ea03e-b684-4059-823f-b67c7d1cf46d None None] MFA Rules not processed for user `8cbb2b875006401cb6276d606d49c5fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:14.411087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-783ea03e-b684-4059-823f-b67c7d1cf46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:14.491829 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1247/2505] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:14 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:14.496477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54186b25-c9c4-4f3e-bf3d-f6cc65af0024 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.547837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:14.559665 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54186b25-c9c4-4f3e-bf3d-f6cc65af0024 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.565527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973dcce-0800-49f0-9b23-841f94a78b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.566127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973dcce-0800-49f0-9b23-841f94a78b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.566405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973dcce-0800-49f0-9b23-841f94a78b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.566678 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973dcce-0800-49f0-9b23-841f94a78b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.567311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973dcce-0800-49f0-9b23-841f94a78b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.569253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973dcce-0800-49f0-9b23-841f94a78b03 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.597095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1259/2506] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:14 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:24:14.601250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ea03e-b684-4059-823f-b67c7d1cf46d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.611976 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ea03e-b684-4059-823f-b67c7d1cf46d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.669073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce3f14-b30b-4eb3-a199-9742d6da178c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.669675 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce3f14-b30b-4eb3-a199-9742d6da178c admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.669951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce3f14-b30b-4eb3-a199-9742d6da178c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.670217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce3f14-b30b-4eb3-a199-9742d6da178c admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.670833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce3f14-b30b-4eb3-a199-9742d6da178c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.673335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce3f14-b30b-4eb3-a199-9742d6da178c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.694947 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1248/2507] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:14 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:14.698805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973dcce-0800-49f0-9b23-841f94a78b03 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.709174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973dcce-0800-49f0-9b23-841f94a78b03 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.714125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8552e5f-8c56-4063-a38a-35eab3101ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.714745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8552e5f-8c56-4063-a38a-35eab3101ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.715021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8552e5f-8c56-4063-a38a-35eab3101ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.715289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8552e5f-8c56-4063-a38a-35eab3101ba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.715973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8552e5f-8c56-4063-a38a-35eab3101ba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.717641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8552e5f-8c56-4063-a38a-35eab3101ba0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.728254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1260/2508] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:14.732292 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce3f14-b30b-4eb3-a199-9742d6da178c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.742848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce3f14-b30b-4eb3-a199-9742d6da178c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.747560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5fdf0-d4a9-4cd7-add4-b5d68d22258c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.748212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5fdf0-d4a9-4cd7-add4-b5d68d22258c admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.748486 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5fdf0-d4a9-4cd7-add4-b5d68d22258c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.748751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5fdf0-d4a9-4cd7-add4-b5d68d22258c admin admin] PATH_INFO: `/v3/projects/4f01e7d932fd4593a01fb08955b388bc/users/602fcca80e574554bc06e2de759353d3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.764158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5fdf0-d4a9-4cd7-add4-b5d68d22258c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f01e7d932fd4593a01fb08955b388bc, user_id=602fcca80e574554bc06e2de759353d3, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.766922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5fdf0-d4a9-4cd7-add4-b5d68d22258c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.778901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1249/2509] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:14 2022] PUT /identity/v3/projects/4f01e7d932fd4593a01fb08955b388bc/users/602fcca80e574554bc06e2de759353d3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:14.782994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8552e5f-8c56-4063-a38a-35eab3101ba0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.792880 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8552e5f-8c56-4063-a38a-35eab3101ba0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.849140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa4e315-7202-4811-887c-7a2b44305b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.849618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4e315-7202-4811-887c-7a2b44305b2e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.849892 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4e315-7202-4811-887c-7a2b44305b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.850169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa4e315-7202-4811-887c-7a2b44305b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.850803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4e315-7202-4811-887c-7a2b44305b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.852424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa4e315-7202-4811-887c-7a2b44305b2e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.862904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1261/2510] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:24:14.867419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5fdf0-d4a9-4cd7-add4-b5d68d22258c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:14.878105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5fdf0-d4a9-4cd7-add4-b5d68d22258c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:14.882883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55b23b-ad38-420a-b9ba-e4a957c4de91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:14.883583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b23b-ad38-420a-b9ba-e4a957c4de91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.883849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b23b-ad38-420a-b9ba-e4a957c4de91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.884121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa55b23b-ad38-420a-b9ba-e4a957c4de91 admin admin] PATH_INFO: `/v3/projects/4f01e7d932fd4593a01fb08955b388bc/users/602fcca80e574554bc06e2de759353d3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.886025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55b23b-ad38-420a-b9ba-e4a957c4de91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f01e7d932fd4593a01fb08955b388bc, user_id=602fcca80e574554bc06e2de759353d3, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:14.888715 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa55b23b-ad38-420a-b9ba-e4a957c4de91 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:14.899713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1250/2511] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:14 2022] PUT /identity/v3/projects/4f01e7d932fd4593a01fb08955b388bc/users/602fcca80e574554bc06e2de759353d3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:14.910487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb064d9d-1141-4620-a630-95705630888a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:14.910945 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb064d9d-1141-4620-a630-95705630888a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:14.911067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb064d9d-1141-4620-a630-95705630888a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:14.939406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fb064d9d-1141-4620-a630-95705630888a None None] MFA Rules not processed for user `602fcca80e574554bc06e2de759353d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:14.940637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fb064d9d-1141-4620-a630-95705630888a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:15.009870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1262/2512] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:15.021949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55b23b-ad38-420a-b9ba-e4a957c4de91 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:15.023804 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-aa55b23b-ad38-420a-b9ba-e4a957c4de91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:15.056704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:15.087248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa55b23b-ad38-420a-b9ba-e4a957c4de91 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:15.090859 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a500d-0cce-443d-ac1a-c004cbf7f980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:15.091480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a500d-0cce-443d-ac1a-c004cbf7f980 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:15.091763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a500d-0cce-443d-ac1a-c004cbf7f980 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:15.092030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850a500d-0cce-443d-ac1a-c004cbf7f980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:15.100927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a500d-0cce-443d-ac1a-c004cbf7f980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:15.102289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850a500d-0cce-443d-ac1a-c004cbf7f980 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:15.111608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1251/2513] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:15 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:15.648077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb064d9d-1141-4620-a630-95705630888a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:15.657883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb064d9d-1141-4620-a630-95705630888a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:15.661483 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0381a2-a3c2-4ab3-b2e5-36a6282d831b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:15.662073 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0381a2-a3c2-4ab3-b2e5-36a6282d831b service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:15.662350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0381a2-a3c2-4ab3-b2e5-36a6282d831b service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:15.662620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0381a2-a3c2-4ab3-b2e5-36a6282d831b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:15.708387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:15.722266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0381a2-a3c2-4ab3-b2e5-36a6282d831b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:15.723754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0381a2-a3c2-4ab3-b2e5-36a6282d831b service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:15.732517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1263/2514] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:19.755281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a500d-0cce-443d-ac1a-c004cbf7f980 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:19.756717 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-850a500d-0cce-443d-ac1a-c004cbf7f980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:19.768376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850a500d-0cce-443d-ac1a-c004cbf7f980 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:19.772013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ae97a-5fde-4f2d-95df-14ed01ba4a72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:19.772597 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ae97a-5fde-4f2d-95df-14ed01ba4a72 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:19.772877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ae97a-5fde-4f2d-95df-14ed01ba4a72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:19.773148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ae97a-5fde-4f2d-95df-14ed01ba4a72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:19.828215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ae97a-5fde-4f2d-95df-14ed01ba4a72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:19.829653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ae97a-5fde-4f2d-95df-14ed01ba4a72 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:19.839420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1252/2515] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:21.924185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0381a2-a3c2-4ab3-b2e5-36a6282d831b service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:21.993570 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0381a2-a3c2-4ab3-b2e5-36a6282d831b service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:21.998148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:21.998835 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:21.999180 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:21.999546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] PATH_INFO: `/v3/users/c83bcebeab5e484690bbcb29a97f165d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:22.000721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c83bcebeab5e484690bbcb29a97f165d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:22.002694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:22.050635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c83bcebeab5e484690bbcb29a97f165d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.059072 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c83bcebeab5e484690bbcb29a97f165d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.059727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c83bcebeab5e484690bbcb29a97f165d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.060354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c83bcebeab5e484690bbcb29a97f165d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.068783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c83bcebeab5e484690bbcb29a97f165d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.087977 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1264/2516] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:21 2022] DELETE /identity/v3/users/c83bcebeab5e484690bbcb29a97f165d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:22.092635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf719580-f5e0-4ec2-9f85-40734607c5d8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:22.093110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf719580-f5e0-4ec2-9f85-40734607c5d8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:22.093268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf719580-f5e0-4ec2-9f85-40734607c5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:22.112387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-cf719580-f5e0-4ec2-9f85-40734607c5d8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:22.113594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-cf719580-f5e0-4ec2-9f85-40734607c5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:22.177638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1253/2517] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:22.187701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:22.189105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:22.251206 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae2b43-f794-444a-8959-8a167a4c7199 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:22.255769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e54ee-fd13-41d6-a93a-88e7828bdf75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:22.256489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e54ee-fd13-41d6-a93a-88e7828bdf75 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:22.256764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e54ee-fd13-41d6-a93a-88e7828bdf75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:22.257032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e54ee-fd13-41d6-a93a-88e7828bdf75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:22.267976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e54ee-fd13-41d6-a93a-88e7828bdf75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:22.270626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e54ee-fd13-41d6-a93a-88e7828bdf75 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:22.281260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1265/2518] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:22.554937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf719580-f5e0-4ec2-9f85-40734607c5d8 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:22.556780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-cf719580-f5e0-4ec2-9f85-40734607c5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:22.569784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf719580-f5e0-4ec2-9f85-40734607c5d8 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:22.574787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:22.575704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:22.576033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:22.576342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] PATH_INFO: `/v3/projects/8bacbd02c499422d9d8982c361550de1` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:22.577502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bacbd02c499422d9d8982c361550de1)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:22.579642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:22.628864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bacbd02c499422d9d8982c361550de1', 'request_id': 'req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.636606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bacbd02c499422d9d8982c361550de1', 'request_id': 'req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.642136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bacbd02c499422d9d8982c361550de1', 'request_id': 'req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.643217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:22.656558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] The token cache is being invalidate because project 8bacbd02c499422d9d8982c361550de1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:22.656842 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.657468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.658943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1254/2519] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:22 2022] DELETE /identity/v3/projects/8bacbd02c499422d9d8982c361550de1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:22.663581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e54ee-fd13-41d6-a93a-88e7828bdf75 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:22.672972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e54ee-fd13-41d6-a93a-88e7828bdf75 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:22.676410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:22.731055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:22.731713 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:22.731988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:22.732281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] PATH_INFO: `/v3/users/8cbb2b875006401cb6276d606d49c5fa` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:22.733392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbb2b875006401cb6276d606d49c5fa)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:22.735350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:22.780820 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbb2b875006401cb6276d606d49c5fa'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.789455 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbb2b875006401cb6276d606d49c5fa'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.790146 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbb2b875006401cb6276d606d49c5fa'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.790819 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbb2b875006401cb6276d606d49c5fa'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.798152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbb2b875006401cb6276d606d49c5fa'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:22.815476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1266/2520] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:22 2022] DELETE /identity/v3/users/8cbb2b875006401cb6276d606d49c5fa => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:24:23.096383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:23.097844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:23.111562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6e858c-1ebf-4080-8bad-7d0cf76d0f1a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:23.169058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:23.169691 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:23.169971 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:23.170244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] PATH_INFO: `/v3/projects/e9f458c36b9b41a9af3e0bc920e7d550` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:23.171356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f458c36b9b41a9af3e0bc920e7d550)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:23.173369 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:23.223645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f458c36b9b41a9af3e0bc920e7d550', 'request_id': 'req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.230909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f458c36b9b41a9af3e0bc920e7d550', 'request_id': 'req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.236378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f458c36b9b41a9af3e0bc920e7d550', 'request_id': 'req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.248160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] The token cache is being invalidate because project e9f458c36b9b41a9af3e0bc920e7d550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:23.248452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.249088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.250563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1255/2521] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:23 2022] DELETE /identity/v3/projects/e9f458c36b9b41a9af3e0bc920e7d550 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:23.255694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:23.257120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:23.269585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec1bf4d-3946-4280-ab29-4a45c8d98b58 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:23.335304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:23.335988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:23.336270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:23.336546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] PATH_INFO: `/v3/users/602fcca80e574554bc06e2de759353d3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:23.337699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] RBAC: Authorizing `identity:delete_user(user_id=602fcca80e574554bc06e2de759353d3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:23.339677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:23.372754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:23.393214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '602fcca80e574554bc06e2de759353d3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.401604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '602fcca80e574554bc06e2de759353d3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.402365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '602fcca80e574554bc06e2de759353d3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.403010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '602fcca80e574554bc06e2de759353d3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.415782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-0eb9aa2c-f78b-413e-a97c-9c98284bb059 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '602fcca80e574554bc06e2de759353d3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.438247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1267/2522] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:23 2022] DELETE /identity/v3/users/602fcca80e574554bc06e2de759353d3 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:23.736928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:23.738348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:23.751591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fade6f-1329-43c7-96ff-cf4b2841e0c7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:23.809266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:23.809892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:23.810169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:23.810436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] PATH_INFO: `/v3/projects/4f01e7d932fd4593a01fb08955b388bc` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:23.811582 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f01e7d932fd4593a01fb08955b388bc)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:23.813487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:23.844664 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:23.862025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f01e7d932fd4593a01fb08955b388bc', 'request_id': 'req-adf915fb-60cc-4605-9543-8833ed65abb7'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.869427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f01e7d932fd4593a01fb08955b388bc', 'request_id': 'req-adf915fb-60cc-4605-9543-8833ed65abb7'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.874980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f01e7d932fd4593a01fb08955b388bc', 'request_id': 'req-adf915fb-60cc-4605-9543-8833ed65abb7'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.886759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] The token cache is being invalidate because project 4f01e7d932fd4593a01fb08955b388bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:23.887043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.887696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-adf915fb-60cc-4605-9543-8833ed65abb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:23.889157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1256/2523] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:23 2022] DELETE /identity/v3/projects/4f01e7d932fd4593a01fb08955b388bc => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:23.895571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08240e49-b039-4309-97d9-4b1da45b23f3 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:23.895853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08240e49-b039-4309-97d9-4b1da45b23f3 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:23.896126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08240e49-b039-4309-97d9-4b1da45b23f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:23.915717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-08240e49-b039-4309-97d9-4b1da45b23f3 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:23.917104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-08240e49-b039-4309-97d9-4b1da45b23f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:23.976940 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1268/2524] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:23.985324 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acda0ae-03a8-4d64-a858-3353b4ddb819 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:23.985603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acda0ae-03a8-4d64-a858-3353b4ddb819 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:23.985870 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acda0ae-03a8-4d64-a858-3353b4ddb819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.006117 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7acda0ae-03a8-4d64-a858-3353b4ddb819 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:24.007310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7acda0ae-03a8-4d64-a858-3353b4ddb819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:24.015306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1257/2525] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:24.019360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08240e49-b039-4309-97d9-4b1da45b23f3 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.029952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08240e49-b039-4309-97d9-4b1da45b23f3 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.034457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a915811-75e5-4dbf-b191-547d1094c5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.035040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a915811-75e5-4dbf-b191-547d1094c5ed admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.035309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a915811-75e5-4dbf-b191-547d1094c5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.036171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a915811-75e5-4dbf-b191-547d1094c5ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.036816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a915811-75e5-4dbf-b191-547d1094c5ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:24:24.037106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a915811-75e5-4dbf-b191-547d1094c5ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.038693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a915811-75e5-4dbf-b191-547d1094c5ed admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.049906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1269/2526] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:24:24.054144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acda0ae-03a8-4d64-a858-3353b4ddb819 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.063581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acda0ae-03a8-4d64-a858-3353b4ddb819 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.068470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8b53a-7167-4213-9437-1805e70de88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.069061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8b53a-7167-4213-9437-1805e70de88e admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.069339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8b53a-7167-4213-9437-1805e70de88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.069609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b8b53a-7167-4213-9437-1805e70de88e admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.070274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8b53a-7167-4213-9437-1805e70de88e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.072281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b8b53a-7167-4213-9437-1805e70de88e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.101807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1258/2527] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:24 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:24.106038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a915811-75e5-4dbf-b191-547d1094c5ed admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.116453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a915811-75e5-4dbf-b191-547d1094c5ed admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.172610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124462fe-f1be-4cb0-b1f8-3a679a8bbad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.173204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124462fe-f1be-4cb0-b1f8-3a679a8bbad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.173482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124462fe-f1be-4cb0-b1f8-3a679a8bbad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.173754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124462fe-f1be-4cb0-b1f8-3a679a8bbad1 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.174384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124462fe-f1be-4cb0-b1f8-3a679a8bbad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.176592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124462fe-f1be-4cb0-b1f8-3a679a8bbad1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.197265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1270/2528] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:24 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:24.201595 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8b53a-7167-4213-9437-1805e70de88e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.212404 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b8b53a-7167-4213-9437-1805e70de88e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.216970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbf6c1-141a-4a9d-8b0a-424da8e964ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.217558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbf6c1-141a-4a9d-8b0a-424da8e964ec admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.217833 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbf6c1-141a-4a9d-8b0a-424da8e964ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.218100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cbf6c1-141a-4a9d-8b0a-424da8e964ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.218735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbf6c1-141a-4a9d-8b0a-424da8e964ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.220366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cbf6c1-141a-4a9d-8b0a-424da8e964ec admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.230697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1259/2529] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:24.234431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124462fe-f1be-4cb0-b1f8-3a679a8bbad1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.244593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124462fe-f1be-4cb0-b1f8-3a679a8bbad1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.249190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a4e713-ed6a-4d2c-ab02-f4fc5794f606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.249777 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4e713-ed6a-4d2c-ab02-f4fc5794f606 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.250056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4e713-ed6a-4d2c-ab02-f4fc5794f606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.250329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a4e713-ed6a-4d2c-ab02-f4fc5794f606 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.250965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4e713-ed6a-4d2c-ab02-f4fc5794f606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.252609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a4e713-ed6a-4d2c-ab02-f4fc5794f606 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.263024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1271/2530] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:24.267169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbf6c1-141a-4a9d-8b0a-424da8e964ec admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.277462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cbf6c1-141a-4a9d-8b0a-424da8e964ec admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.282065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4cc41-7fa0-4f45-8785-0e73ba2c00ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.282703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4cc41-7fa0-4f45-8785-0e73ba2c00ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.282977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4cc41-7fa0-4f45-8785-0e73ba2c00ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.283246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4cc41-7fa0-4f45-8785-0e73ba2c00ba admin admin] PATH_INFO: `/v3/projects/5b6fca8c6ce146d28f9afd7e2fd789a4/users/2af6ef492e0d4d2da5e7f9fdb5e2e494/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.298422 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4cc41-7fa0-4f45-8785-0e73ba2c00ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b6fca8c6ce146d28f9afd7e2fd789a4, user_id=2af6ef492e0d4d2da5e7f9fdb5e2e494, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.301117 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4cc41-7fa0-4f45-8785-0e73ba2c00ba admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.313699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1260/2531] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:24 2022] PUT /identity/v3/projects/5b6fca8c6ce146d28f9afd7e2fd789a4/users/2af6ef492e0d4d2da5e7f9fdb5e2e494/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:24.325604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d92f34-b3cc-4d57-9aae-e98c2e6e9522 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.325886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d92f34-b3cc-4d57-9aae-e98c2e6e9522 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.326160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d92f34-b3cc-4d57-9aae-e98c2e6e9522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.347944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:24.356265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-c7d92f34-b3cc-4d57-9aae-e98c2e6e9522 None None] MFA Rules not processed for user `2af6ef492e0d4d2da5e7f9fdb5e2e494`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:24.358016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c7d92f34-b3cc-4d57-9aae-e98c2e6e9522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:24.421652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1272/2532] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:24.427365 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4cc41-7fa0-4f45-8785-0e73ba2c00ba admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.489314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4cc41-7fa0-4f45-8785-0e73ba2c00ba admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.494037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64442efd-c6c4-44c5-a0c0-9e63ee5f52d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.494628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64442efd-c6c4-44c5-a0c0-9e63ee5f52d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.494902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64442efd-c6c4-44c5-a0c0-9e63ee5f52d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.495168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64442efd-c6c4-44c5-a0c0-9e63ee5f52d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.495822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64442efd-c6c4-44c5-a0c0-9e63ee5f52d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.497705 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64442efd-c6c4-44c5-a0c0-9e63ee5f52d6 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.524513 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1261/2533] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:24 2022] POST /identity/v3/projects => generated 375 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:24.528640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d92f34-b3cc-4d57-9aae-e98c2e6e9522 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.540352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d92f34-b3cc-4d57-9aae-e98c2e6e9522 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.596147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00bb70-f408-4473-87c6-c79b94834c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.596746 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00bb70-f408-4473-87c6-c79b94834c7d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.597025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00bb70-f408-4473-87c6-c79b94834c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.597292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00bb70-f408-4473-87c6-c79b94834c7d admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.597904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00bb70-f408-4473-87c6-c79b94834c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.600690 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d00bb70-f408-4473-87c6-c79b94834c7d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.622235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1273/2534] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:24 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:24.626174 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64442efd-c6c4-44c5-a0c0-9e63ee5f52d6 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.636355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64442efd-c6c4-44c5-a0c0-9e63ee5f52d6 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.640969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e549780-664a-4f63-abb1-f988eb86a771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.641562 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e549780-664a-4f63-abb1-f988eb86a771 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.641839 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e549780-664a-4f63-abb1-f988eb86a771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.642109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e549780-664a-4f63-abb1-f988eb86a771 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.642748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e549780-664a-4f63-abb1-f988eb86a771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.644418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e549780-664a-4f63-abb1-f988eb86a771 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.655231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1262/2535] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:24.659236 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00bb70-f408-4473-87c6-c79b94834c7d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.669947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00bb70-f408-4473-87c6-c79b94834c7d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.674452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b73878-748f-4ea6-8090-73e3b68a538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.675081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b73878-748f-4ea6-8090-73e3b68a538b admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.675354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b73878-748f-4ea6-8090-73e3b68a538b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.675650 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b73878-748f-4ea6-8090-73e3b68a538b admin admin] PATH_INFO: `/v3/projects/04910ba407814c56a686ec4ca86026e9/users/2e620613383748098d07f24526acd481/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.691255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b73878-748f-4ea6-8090-73e3b68a538b admin admin] RBAC: Authorizing `identity:create_grant(project_id=04910ba407814c56a686ec4ca86026e9, user_id=2e620613383748098d07f24526acd481, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.694875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b73878-748f-4ea6-8090-73e3b68a538b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.707198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1274/2536] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:24 2022] PUT /identity/v3/projects/04910ba407814c56a686ec4ca86026e9/users/2e620613383748098d07f24526acd481/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:24.711918 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e549780-664a-4f63-abb1-f988eb86a771 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.721751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e549780-664a-4f63-abb1-f988eb86a771 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.742114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:24.781674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c77bb9-01dc-4b5b-b275-97573f567144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.782274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c77bb9-01dc-4b5b-b275-97573f567144 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.782548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c77bb9-01dc-4b5b-b275-97573f567144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.782813 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c77bb9-01dc-4b5b-b275-97573f567144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.783467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c77bb9-01dc-4b5b-b275-97573f567144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.785051 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c77bb9-01dc-4b5b-b275-97573f567144 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.795915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1263/2537] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:24:24.800024 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b73878-748f-4ea6-8090-73e3b68a538b admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.810865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b73878-748f-4ea6-8090-73e3b68a538b admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:24.815402 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12af2a-6c0a-491f-8f79-7c3a2a378d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:24.816069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12af2a-6c0a-491f-8f79-7c3a2a378d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.816343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12af2a-6c0a-491f-8f79-7c3a2a378d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.816609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac12af2a-6c0a-491f-8f79-7c3a2a378d02 admin admin] PATH_INFO: `/v3/projects/04910ba407814c56a686ec4ca86026e9/users/2e620613383748098d07f24526acd481/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.819436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12af2a-6c0a-491f-8f79-7c3a2a378d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04910ba407814c56a686ec4ca86026e9, user_id=2e620613383748098d07f24526acd481, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:24.822113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac12af2a-6c0a-491f-8f79-7c3a2a378d02 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:24.832237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1275/2538] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:24 2022] PUT /identity/v3/projects/04910ba407814c56a686ec4ca86026e9/users/2e620613383748098d07f24526acd481/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:24.843771 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f8462-7263-4582-bdf7-369ee4dcc6f6 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:24.844062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f8462-7263-4582-bdf7-369ee4dcc6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:24.844336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f8462-7263-4582-bdf7-369ee4dcc6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:24.873273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-bc2f8462-7263-4582-bdf7-369ee4dcc6f6 None None] MFA Rules not processed for user `2e620613383748098d07f24526acd481`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:24.874505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bc2f8462-7263-4582-bdf7-369ee4dcc6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:24.949154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1264/2539] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:24 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:24.960360 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12af2a-6c0a-491f-8f79-7c3a2a378d02 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:24.962077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ac12af2a-6c0a-491f-8f79-7c3a2a378d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:25.019385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac12af2a-6c0a-491f-8f79-7c3a2a378d02 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:25.022927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13366a-87ef-4ba1-940e-eb66671dca12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:25.023550 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13366a-87ef-4ba1-940e-eb66671dca12 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:25.023828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13366a-87ef-4ba1-940e-eb66671dca12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:25.024099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13366a-87ef-4ba1-940e-eb66671dca12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:25.029070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:25.080193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13366a-87ef-4ba1-940e-eb66671dca12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:25.081631 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13366a-87ef-4ba1-940e-eb66671dca12 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:25.090544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1276/2540] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:27.642248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f8462-7263-4582-bdf7-369ee4dcc6f6 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:27.718725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f8462-7263-4582-bdf7-369ee4dcc6f6 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:27.727005 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:27.727848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:27.728211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:27.728559 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] PATH_INFO: `/v3/users/2af6ef492e0d4d2da5e7f9fdb5e2e494` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:27.729948 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=2af6ef492e0d4d2da5e7f9fdb5e2e494)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:27.732156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:27.788974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2af6ef492e0d4d2da5e7f9fdb5e2e494'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:27.789626 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2af6ef492e0d4d2da5e7f9fdb5e2e494'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:27.791532 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:27.799568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2af6ef492e0d4d2da5e7f9fdb5e2e494'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:27.809089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2af6ef492e0d4d2da5e7f9fdb5e2e494'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:27.809819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2af6ef492e0d4d2da5e7f9fdb5e2e494'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:27.828191 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1265/2541] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:27 2022] DELETE /identity/v3/users/2af6ef492e0d4d2da5e7f9fdb5e2e494 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:27.833525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c6e66-5e98-4ec7-bdad-5e061c29a062 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:27.834039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c6e66-5e98-4ec7-bdad-5e061c29a062 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:27.834293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c6e66-5e98-4ec7-bdad-5e061c29a062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:27.854100 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-209c6e66-5e98-4ec7-bdad-5e061c29a062 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:27.855311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-209c6e66-5e98-4ec7-bdad-5e061c29a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:27.918335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1277/2542] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:27.929054 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:27.930525 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:27.990847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a4ebb-602b-44d5-9035-8c979791b1ca admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:27.994491 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffd604-7ec7-4882-b45a-ad3ff49dfcb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:27.995098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffd604-7ec7-4882-b45a-ad3ff49dfcb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:27.995375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffd604-7ec7-4882-b45a-ad3ff49dfcb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:27.995678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bffd604-7ec7-4882-b45a-ad3ff49dfcb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:28.005245 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffd604-7ec7-4882-b45a-ad3ff49dfcb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:28.006736 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bffd604-7ec7-4882-b45a-ad3ff49dfcb9 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:28.017475 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1266/2543] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:24:28.286939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c6e66-5e98-4ec7-bdad-5e061c29a062 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:28.288636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-209c6e66-5e98-4ec7-bdad-5e061c29a062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:28.303828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c6e66-5e98-4ec7-bdad-5e061c29a062 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:28.308606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:28.309240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:28.309513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:28.309783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] PATH_INFO: `/v3/projects/5b6fca8c6ce146d28f9afd7e2fd789a4` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:28.311080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b6fca8c6ce146d28f9afd7e2fd789a4)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:28.313285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:28.361764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b6fca8c6ce146d28f9afd7e2fd789a4', 'request_id': 'req-227a4cc3-6e10-401c-8da7-a839a1936e6c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:28.367603 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b6fca8c6ce146d28f9afd7e2fd789a4', 'request_id': 'req-227a4cc3-6e10-401c-8da7-a839a1936e6c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:28.376706 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b6fca8c6ce146d28f9afd7e2fd789a4', 'request_id': 'req-227a4cc3-6e10-401c-8da7-a839a1936e6c'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:28.390209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] The token cache is being invalidate because project 5b6fca8c6ce146d28f9afd7e2fd789a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:28.390598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:28.391231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:28.392829 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1278/2544] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:28 2022] DELETE /identity/v3/projects/5b6fca8c6ce146d28f9afd7e2fd789a4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:28.396971 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffd604-7ec7-4882-b45a-ad3ff49dfcb9 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:28.406933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bffd604-7ec7-4882-b45a-ad3ff49dfcb9 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:28.486810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:28.487623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:28.487909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:28.488362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] PATH_INFO: `/v3/users/2e620613383748098d07f24526acd481` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:28.490098 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e620613383748098d07f24526acd481)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:28.492218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:28.540993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e620613383748098d07f24526acd481'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:28.541681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e620613383748098d07f24526acd481'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:28.549146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e620613383748098d07f24526acd481'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:28.556522 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e620613383748098d07f24526acd481'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:28.557184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-2dfa716b-e854-4f61-bb57-edb71a18b8a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e620613383748098d07f24526acd481'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:28.562610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:INSERT=2,keystone:DELETE=12,keystone:SELECT=86 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:28.580703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1267/2545] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:28 2022] DELETE /identity/v3/users/2e620613383748098d07f24526acd481 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:28.859842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:28.861459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:28.885070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227a4cc3-6e10-401c-8da7-a839a1936e6c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:28.920342 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:28.959630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:28.960270 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:28.960546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:28.960817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] PATH_INFO: `/v3/projects/04910ba407814c56a686ec4ca86026e9` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:28.961966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04910ba407814c56a686ec4ca86026e9)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:28.964086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.013799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04910ba407814c56a686ec4ca86026e9', 'request_id': 'req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:29.020554 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04910ba407814c56a686ec4ca86026e9', 'request_id': 'req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:29.028986 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04910ba407814c56a686ec4ca86026e9', 'request_id': 'req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:29.041204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] The token cache is being invalidate because project 04910ba407814c56a686ec4ca86026e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:29.041553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:29.042222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f5dc05b-6d50-462b-8d0b-6e35dcdbc2e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:29.043962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1279/2546] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:28 2022] DELETE /identity/v3/projects/04910ba407814c56a686ec4ca86026e9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:29.049210 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fe12a-12aa-4031-86c4-e3a223f1ef0c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.049754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fe12a-12aa-4031-86c4-e3a223f1ef0c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.049754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615fe12a-12aa-4031-86c4-e3a223f1ef0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.070713 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-615fe12a-12aa-4031-86c4-e3a223f1ef0c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:29.072010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-615fe12a-12aa-4031-86c4-e3a223f1ef0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:29.130735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1268/2547] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:29.138356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16774f1-5f13-484e-afdd-1c14768655fc None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.138641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16774f1-5f13-484e-afdd-1c14768655fc None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.138919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16774f1-5f13-484e-afdd-1c14768655fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.159948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b16774f1-5f13-484e-afdd-1c14768655fc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:29.161998 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b16774f1-5f13-484e-afdd-1c14768655fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:29.169803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1280/2548] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:29.173903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fe12a-12aa-4031-86c4-e3a223f1ef0c None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.183818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615fe12a-12aa-4031-86c4-e3a223f1ef0c None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.188326 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57962c-9826-4e51-8d63-d3a39f1d0d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.188909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57962c-9826-4e51-8d63-d3a39f1d0d45 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.189179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57962c-9826-4e51-8d63-d3a39f1d0d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.189444 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f57962c-9826-4e51-8d63-d3a39f1d0d45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.190080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57962c-9826-4e51-8d63-d3a39f1d0d45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:24:29.190374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57962c-9826-4e51-8d63-d3a39f1d0d45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.192367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f57962c-9826-4e51-8d63-d3a39f1d0d45 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.203534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1269/2549] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:24:29.208182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16774f1-5f13-484e-afdd-1c14768655fc None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.217230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16774f1-5f13-484e-afdd-1c14768655fc None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.221773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a764cc-fc8d-4a9f-ad94-a45206c09d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.222347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a764cc-fc8d-4a9f-ad94-a45206c09d77 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.222619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a764cc-fc8d-4a9f-ad94-a45206c09d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.223586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a764cc-fc8d-4a9f-ad94-a45206c09d77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.224323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a764cc-fc8d-4a9f-ad94-a45206c09d77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.226240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a764cc-fc8d-4a9f-ad94-a45206c09d77 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.252726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1281/2550] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:29 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:29.257653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57962c-9826-4e51-8d63-d3a39f1d0d45 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.267629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f57962c-9826-4e51-8d63-d3a39f1d0d45 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.322720 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b69524-6806-40a3-b16f-4c60852772f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.323207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b69524-6806-40a3-b16f-4c60852772f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.323506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b69524-6806-40a3-b16f-4c60852772f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.323780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b69524-6806-40a3-b16f-4c60852772f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.324402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b69524-6806-40a3-b16f-4c60852772f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.326336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b69524-6806-40a3-b16f-4c60852772f1 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.348531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1270/2551] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:29 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:29.352754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a764cc-fc8d-4a9f-ad94-a45206c09d77 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.365924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a764cc-fc8d-4a9f-ad94-a45206c09d77 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.370723 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706b32e-148e-48cf-a704-5b117d8253ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.371315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706b32e-148e-48cf-a704-5b117d8253ac admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.371623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706b32e-148e-48cf-a704-5b117d8253ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.371894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706b32e-148e-48cf-a704-5b117d8253ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.372531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706b32e-148e-48cf-a704-5b117d8253ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.374163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706b32e-148e-48cf-a704-5b117d8253ac admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.385687 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1282/2552] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:29.389917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b69524-6806-40a3-b16f-4c60852772f1 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.400973 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b69524-6806-40a3-b16f-4c60852772f1 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.405602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a81fa7-935e-4dc7-8b67-428b8e2789e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.406187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a81fa7-935e-4dc7-8b67-428b8e2789e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.406460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a81fa7-935e-4dc7-8b67-428b8e2789e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.406744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a81fa7-935e-4dc7-8b67-428b8e2789e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.407353 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a81fa7-935e-4dc7-8b67-428b8e2789e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.408984 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a81fa7-935e-4dc7-8b67-428b8e2789e2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.419962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1271/2553] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:29.424790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706b32e-148e-48cf-a704-5b117d8253ac admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.435430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706b32e-148e-48cf-a704-5b117d8253ac admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.440016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250560a-2b46-424b-bc81-b412726705bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.440661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250560a-2b46-424b-bc81-b412726705bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.440937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250560a-2b46-424b-bc81-b412726705bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.441209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250560a-2b46-424b-bc81-b412726705bf admin admin] PATH_INFO: `/v3/projects/97e6e815f17446ca846f5e162df7936d/users/00e107b8a45b4fdfa2cea64ab42180e0/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.456559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250560a-2b46-424b-bc81-b412726705bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e6e815f17446ca846f5e162df7936d, user_id=00e107b8a45b4fdfa2cea64ab42180e0, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.459879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250560a-2b46-424b-bc81-b412726705bf admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.472807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1283/2554] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:29 2022] PUT /identity/v3/projects/97e6e815f17446ca846f5e162df7936d/users/00e107b8a45b4fdfa2cea64ab42180e0/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:29.485366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e530f-9fb8-4fd7-b689-b67001a4ce8d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.485648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e530f-9fb8-4fd7-b689-b67001a4ce8d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.485916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e530f-9fb8-4fd7-b689-b67001a4ce8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.513988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-f34e530f-9fb8-4fd7-b689-b67001a4ce8d None None] MFA Rules not processed for user `00e107b8a45b4fdfa2cea64ab42180e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:29.515172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f34e530f-9fb8-4fd7-b689-b67001a4ce8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:29.556762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:29.581253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1272/2555] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:29.585960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250560a-2b46-424b-bc81-b412726705bf admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.643516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:29.648107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250560a-2b46-424b-bc81-b412726705bf admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.654134 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ecb3e-df94-4526-b53f-546774f28bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.654888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ecb3e-df94-4526-b53f-546774f28bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.655365 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ecb3e-df94-4526-b53f-546774f28bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.655791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ecb3e-df94-4526-b53f-546774f28bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.656577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ecb3e-df94-4526-b53f-546774f28bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.658482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ecb3e-df94-4526-b53f-546774f28bc4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.684669 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1284/2556] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:29 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:29.688802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e530f-9fb8-4fd7-b689-b67001a4ce8d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.699783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e530f-9fb8-4fd7-b689-b67001a4ce8d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.755284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f8a4-e1a8-446a-9cf4-3773976dbc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.755906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f8a4-e1a8-446a-9cf4-3773976dbc15 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.756182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f8a4-e1a8-446a-9cf4-3773976dbc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.756454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6049f8a4-e1a8-446a-9cf4-3773976dbc15 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.757081 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049f8a4-e1a8-446a-9cf4-3773976dbc15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.758982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6049f8a4-e1a8-446a-9cf4-3773976dbc15 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.783941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1273/2557] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:29 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:29.785235 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ecb3e-df94-4526-b53f-546774f28bc4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.795952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ecb3e-df94-4526-b53f-546774f28bc4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.800525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e5eb9-281d-4d3f-9412-5386cb43bd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.801717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e5eb9-281d-4d3f-9412-5386cb43bd9a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.801996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e5eb9-281d-4d3f-9412-5386cb43bd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.802264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e5eb9-281d-4d3f-9412-5386cb43bd9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.802900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e5eb9-281d-4d3f-9412-5386cb43bd9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.804573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e5eb9-281d-4d3f-9412-5386cb43bd9a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.814882 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1285/2558] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:29.819378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f8a4-e1a8-446a-9cf4-3773976dbc15 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.829248 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6049f8a4-e1a8-446a-9cf4-3773976dbc15 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.834134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e825f59-dfe8-480e-9ac4-44258e8c2bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.834671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e825f59-dfe8-480e-9ac4-44258e8c2bba admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.834949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e825f59-dfe8-480e-9ac4-44258e8c2bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.835219 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e825f59-dfe8-480e-9ac4-44258e8c2bba admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.835893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e825f59-dfe8-480e-9ac4-44258e8c2bba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.837496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e825f59-dfe8-480e-9ac4-44258e8c2bba admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.847196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1274/2559] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:29 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:29.851385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e5eb9-281d-4d3f-9412-5386cb43bd9a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:29.861164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e5eb9-281d-4d3f-9412-5386cb43bd9a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:29.866630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abefc0a-0d93-4062-976b-5f678f9c09b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:29.867260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abefc0a-0d93-4062-976b-5f678f9c09b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.867585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abefc0a-0d93-4062-976b-5f678f9c09b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.867860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abefc0a-0d93-4062-976b-5f678f9c09b8 admin admin] PATH_INFO: `/v3/projects/13cf0f607293448c9bf2817d953aa18c/users/c0411d53fe85439b8ddf02ca4dc4419f/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.882914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abefc0a-0d93-4062-976b-5f678f9c09b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13cf0f607293448c9bf2817d953aa18c, user_id=c0411d53fe85439b8ddf02ca4dc4419f, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:29.885619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abefc0a-0d93-4062-976b-5f678f9c09b8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:29.898313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1286/2560] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:29 2022] PUT /identity/v3/projects/13cf0f607293448c9bf2817d953aa18c/users/c0411d53fe85439b8ddf02ca4dc4419f/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:29.909591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53241f-9426-409a-9e97-909c4be71946 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:29.909876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53241f-9426-409a-9e97-909c4be71946 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:29.910214 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53241f-9426-409a-9e97-909c4be71946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:29.939121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-bd53241f-9426-409a-9e97-909c4be71946 None None] MFA Rules not processed for user `c0411d53fe85439b8ddf02ca4dc4419f`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:29.940345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bd53241f-9426-409a-9e97-909c4be71946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:30.005722 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1275/2561] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:30.010240 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abefc0a-0d93-4062-976b-5f678f9c09b8 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:30.073457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abefc0a-0d93-4062-976b-5f678f9c09b8 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:30.078131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e8b82-b4b7-44cc-8f74-a1294840b743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:30.078765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e8b82-b4b7-44cc-8f74-a1294840b743 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:30.079238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e8b82-b4b7-44cc-8f74-a1294840b743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:30.079548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e8b82-b4b7-44cc-8f74-a1294840b743 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:30.080182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e8b82-b4b7-44cc-8f74-a1294840b743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:30.082242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714e8b82-b4b7-44cc-8f74-a1294840b743 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:30.114733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1287/2562] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:30 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:30.116007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53241f-9426-409a-9e97-909c4be71946 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:30.129354 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53241f-9426-409a-9e97-909c4be71946 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:30.181575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:30.192280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd4c00-7c79-46a9-9b6f-dbcc5e927080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:30.192889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd4c00-7c79-46a9-9b6f-dbcc5e927080 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:30.193167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd4c00-7c79-46a9-9b6f-dbcc5e927080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:30.193439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd4c00-7c79-46a9-9b6f-dbcc5e927080 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:30.194070 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd4c00-7c79-46a9-9b6f-dbcc5e927080 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:30.196058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd4c00-7c79-46a9-9b6f-dbcc5e927080 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:30.220636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1276/2563] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:30 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:30.222148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e8b82-b4b7-44cc-8f74-a1294840b743 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:30.234887 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e8b82-b4b7-44cc-8f74-a1294840b743 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:30.240105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61012d-a16f-4db5-ae01-0d3d86863238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:30.240687 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61012d-a16f-4db5-ae01-0d3d86863238 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:30.240994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61012d-a16f-4db5-ae01-0d3d86863238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:30.241281 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e61012d-a16f-4db5-ae01-0d3d86863238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:30.241976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61012d-a16f-4db5-ae01-0d3d86863238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:30.243671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e61012d-a16f-4db5-ae01-0d3d86863238 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:30.258083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1288/2564] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:30.259316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd4c00-7c79-46a9-9b6f-dbcc5e927080 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:30.269783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd4c00-7c79-46a9-9b6f-dbcc5e927080 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:30.275733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd373064-6428-4bb0-82a6-25c4feba7a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:30.275733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd373064-6428-4bb0-82a6-25c4feba7a58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:30.275733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd373064-6428-4bb0-82a6-25c4feba7a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:30.276186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd373064-6428-4bb0-82a6-25c4feba7a58 admin admin] PATH_INFO: `/v3/projects/4e2e4361992b49959c4426d9d5aa9670/users/646b64a19dc641f094b0e84357296faf/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:30.292417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd373064-6428-4bb0-82a6-25c4feba7a58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2e4361992b49959c4426d9d5aa9670, user_id=646b64a19dc641f094b0e84357296faf, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:30.295077 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd373064-6428-4bb0-82a6-25c4feba7a58 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:30.308275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1277/2565] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:30 2022] PUT /identity/v3/projects/4e2e4361992b49959c4426d9d5aa9670/users/646b64a19dc641f094b0e84357296faf/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:30.312219 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61012d-a16f-4db5-ae01-0d3d86863238 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:30.322799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e61012d-a16f-4db5-ae01-0d3d86863238 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:30.378147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ebf82-a101-4e9f-b1e3-5a6a0bbeab90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:30.378693 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ebf82-a101-4e9f-b1e3-5a6a0bbeab90 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:30.379632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ebf82-a101-4e9f-b1e3-5a6a0bbeab90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:30.379906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ebf82-a101-4e9f-b1e3-5a6a0bbeab90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:30.380568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ebf82-a101-4e9f-b1e3-5a6a0bbeab90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:30.382164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ebf82-a101-4e9f-b1e3-5a6a0bbeab90 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:30.391677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1289/2566] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:30.396139 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd373064-6428-4bb0-82a6-25c4feba7a58 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:30.406186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd373064-6428-4bb0-82a6-25c4feba7a58 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:30.411194 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dc7b8-b45e-4433-8793-3b438aec4764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:30.411868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dc7b8-b45e-4433-8793-3b438aec4764 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:30.412141 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dc7b8-b45e-4433-8793-3b438aec4764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:30.412409 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0dc7b8-b45e-4433-8793-3b438aec4764 admin admin] PATH_INFO: `/v3/projects/4e2e4361992b49959c4426d9d5aa9670/users/646b64a19dc641f094b0e84357296faf/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:30.414306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0dc7b8-b45e-4433-8793-3b438aec4764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e2e4361992b49959c4426d9d5aa9670, user_id=646b64a19dc641f094b0e84357296faf, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:30.416967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0dc7b8-b45e-4433-8793-3b438aec4764 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:30.427622 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1278/2567] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:30 2022] PUT /identity/v3/projects/4e2e4361992b49959c4426d9d5aa9670/users/646b64a19dc641f094b0e84357296faf/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:24:30.437744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9f0f7-3937-492b-b46f-f3fe9630f135 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:30.438029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9f0f7-3937-492b-b46f-f3fe9630f135 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:30.438302 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9f0f7-3937-492b-b46f-f3fe9630f135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:30.467636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b9d9f0f7-3937-492b-b46f-f3fe9630f135 None None] MFA Rules not processed for user `646b64a19dc641f094b0e84357296faf`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:30.468877 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b9d9f0f7-3937-492b-b46f-f3fe9630f135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:30.474102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:30.537841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1290/2568] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:30 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:30.549075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dc7b8-b45e-4433-8793-3b438aec4764 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:30.551489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9c0dc7b8-b45e-4433-8793-3b438aec4764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:30.614252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0dc7b8-b45e-4433-8793-3b438aec4764 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:30.618430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb455490-23f1-4845-9e03-5ec1e8a53275 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:30.619024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb455490-23f1-4845-9e03-5ec1e8a53275 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:30.619298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb455490-23f1-4845-9e03-5ec1e8a53275 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:30.619605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb455490-23f1-4845-9e03-5ec1e8a53275 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:30.679297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb455490-23f1-4845-9e03-5ec1e8a53275 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:30.680769 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb455490-23f1-4845-9e03-5ec1e8a53275 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:30.690605 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1279/2569] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:30 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:32.207927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9f0f7-3937-492b-b46f-f3fe9630f135 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:32.217686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9f0f7-3937-492b-b46f-f3fe9630f135 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:32.221249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb29975-b3c1-4612-b47b-2ab383a4035a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:32.221972 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb29975-b3c1-4612-b47b-2ab383a4035a service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:32.222256 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb29975-b3c1-4612-b47b-2ab383a4035a service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:32.223140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb29975-b3c1-4612-b47b-2ab383a4035a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:32.279834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb29975-b3c1-4612-b47b-2ab383a4035a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:32.281207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb29975-b3c1-4612-b47b-2ab383a4035a service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:32.289990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1291/2570] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:32.664541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb455490-23f1-4845-9e03-5ec1e8a53275 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:32.666226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-eb455490-23f1-4845-9e03-5ec1e8a53275 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:32.727595 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb455490-23f1-4845-9e03-5ec1e8a53275 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:32.731414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0563962-42a6-4489-84c7-3fa0916fe614 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:32.732083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0563962-42a6-4489-84c7-3fa0916fe614 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:32.732355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0563962-42a6-4489-84c7-3fa0916fe614 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:32.732620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0563962-42a6-4489-84c7-3fa0916fe614 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:32.734387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c0563962-42a6-4489-84c7-3fa0916fe614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:32.745643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:32.801533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0563962-42a6-4489-84c7-3fa0916fe614 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:32.803037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0563962-42a6-4489-84c7-3fa0916fe614 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:32.812971 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1280/2571] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:32.844107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb29975-b3c1-4612-b47b-2ab383a4035a service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:32.852847 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb29975-b3c1-4612-b47b-2ab383a4035a service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:32.856659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bca693-7952-4092-86b8-f3493898740a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:32.857254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bca693-7952-4092-86b8-f3493898740a service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:32.857532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bca693-7952-4092-86b8-f3493898740a service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:32.857803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bca693-7952-4092-86b8-f3493898740a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:32.867732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bca693-7952-4092-86b8-f3493898740a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:32.869177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bca693-7952-4092-86b8-f3493898740a service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:32.879192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1292/2572] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:32 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:36.049395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0563962-42a6-4489-84c7-3fa0916fe614 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:36.051101 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c0563962-42a6-4489-84c7-3fa0916fe614 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:36.064693 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0563962-42a6-4489-84c7-3fa0916fe614 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:36.071400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:36.072071 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.072351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.072625 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] PATH_INFO: `/v3/users/db2f1171c5394a1c8879cd09c1e685b1` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.088703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] RBAC: Authorizing `identity:delete_user(user_id=db2f1171c5394a1c8879cd09c1e685b1)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:36.090649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:36.135522 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db2f1171c5394a1c8879cd09c1e685b1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.136141 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db2f1171c5394a1c8879cd09c1e685b1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.144180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db2f1171c5394a1c8879cd09c1e685b1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.151927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db2f1171c5394a1c8879cd09c1e685b1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.152629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bc27da2f-bdaf-4ab0-8fcb-979c40d13cff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db2f1171c5394a1c8879cd09c1e685b1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.170115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1281/2573] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:36 2022] DELETE /identity/v3/users/db2f1171c5394a1c8879cd09c1e685b1 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:36.343320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bca693-7952-4092-86b8-f3493898740a service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:36.354497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bca693-7952-4092-86b8-f3493898740a service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:36.412616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:36.413155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.413425 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.413689 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] PATH_INFO: `/v3/projects/68f11e95e757488fa7c4c156194d24ff` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.423827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f11e95e757488fa7c4c156194d24ff)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:36.426374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:36.448424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:36.471052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f11e95e757488fa7c4c156194d24ff', 'request_id': 'req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.476680 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f11e95e757488fa7c4c156194d24ff', 'request_id': 'req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.483999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f11e95e757488fa7c4c156194d24ff', 'request_id': 'req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.496546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] The token cache is being invalidate because project 68f11e95e757488fa7c4c156194d24ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:36.496848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.497493 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-d3a3bb93-7c31-452c-a1ee-045b5d3002ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.499012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1293/2574] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:36 2022] DELETE /identity/v3/projects/68f11e95e757488fa7c4c156194d24ff => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:36.504963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5eb7a3-5115-4da6-9717-aa96f98e283b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.505364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5eb7a3-5115-4da6-9717-aa96f98e283b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.505641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5eb7a3-5115-4da6-9717-aa96f98e283b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.526568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-2e5eb7a3-5115-4da6-9717-aa96f98e283b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:36.527861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2e5eb7a3-5115-4da6-9717-aa96f98e283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:36.593791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1282/2575] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:36.602658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebb7a8-ed39-4dbd-ae3d-baa176a74944 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.603204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebb7a8-ed39-4dbd-ae3d-baa176a74944 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.603511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ebb7a8-ed39-4dbd-ae3d-baa176a74944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.624426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-32ebb7a8-ed39-4dbd-ae3d-baa176a74944 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:36.625637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-32ebb7a8-ed39-4dbd-ae3d-baa176a74944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:36.635665 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1294/2576] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:36.640293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5eb7a3-5115-4da6-9717-aa96f98e283b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:36.651308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5eb7a3-5115-4da6-9717-aa96f98e283b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:36.655987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a33ba4-4fe4-438b-8a1f-1f820a703f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:36.656593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a33ba4-4fe4-438b-8a1f-1f820a703f22 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.656863 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a33ba4-4fe4-438b-8a1f-1f820a703f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.657126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a33ba4-4fe4-438b-8a1f-1f820a703f22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.657905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a33ba4-4fe4-438b-8a1f-1f820a703f22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:24:36.658034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a33ba4-4fe4-438b-8a1f-1f820a703f22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:36.659789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a33ba4-4fe4-438b-8a1f-1f820a703f22 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:36.663316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:36.674204 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1283/2577] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:24:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:24:36.678626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebb7a8-ed39-4dbd-ae3d-baa176a74944 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:36.693765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ebb7a8-ed39-4dbd-ae3d-baa176a74944 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:36.698511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5c049-cbd7-4c51-b765-3a612aa92f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:36.699107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5c049-cbd7-4c51-b765-3a612aa92f23 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.699382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5c049-cbd7-4c51-b765-3a612aa92f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.699694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c5c049-cbd7-4c51-b765-3a612aa92f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.700330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5c049-cbd7-4c51-b765-3a612aa92f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:36.702230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c5c049-cbd7-4c51-b765-3a612aa92f23 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:36.735903 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1295/2578] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:36 2022] POST /identity/v3/projects => generated 403 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:36.737293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a33ba4-4fe4-438b-8a1f-1f820a703f22 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:36.747900 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a33ba4-4fe4-438b-8a1f-1f820a703f22 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:36.816241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb579b8-aa06-49d9-8af7-93f50db122f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:36.817026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb579b8-aa06-49d9-8af7-93f50db122f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.817437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb579b8-aa06-49d9-8af7-93f50db122f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.817931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb579b8-aa06-49d9-8af7-93f50db122f3 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.818902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb579b8-aa06-49d9-8af7-93f50db122f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:36.821677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb579b8-aa06-49d9-8af7-93f50db122f3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:36.846176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1284/2579] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:36 2022] POST /identity/v3/users => generated 302 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:36.852990 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5c049-cbd7-4c51-b765-3a612aa92f23 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:36.861546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c5c049-cbd7-4c51-b765-3a612aa92f23 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:36.867230 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8e761-1747-4151-a073-c6ccb60540fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:36.867969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8e761-1747-4151-a073-c6ccb60540fe admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.868242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8e761-1747-4151-a073-c6ccb60540fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.868516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8e761-1747-4151-a073-c6ccb60540fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.869144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8e761-1747-4151-a073-c6ccb60540fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:36.870737 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d8e761-1747-4151-a073-c6ccb60540fe admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:36.877036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb579b8-aa06-49d9-8af7-93f50db122f3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:36.879546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-0fb579b8-aa06-49d9-8af7-93f50db122f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:36.882176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1296/2580] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:36.886250 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8e761-1747-4151-a073-c6ccb60540fe admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:36.893066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb579b8-aa06-49d9-8af7-93f50db122f3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:36.897166 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d8e761-1747-4151-a073-c6ccb60540fe admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:36.898875 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:36.899639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.899916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.900187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] PATH_INFO: `/v3/users/00e107b8a45b4fdfa2cea64ab42180e0` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.901291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] RBAC: Authorizing `identity:delete_user(user_id=00e107b8a45b4fdfa2cea64ab42180e0)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:36.901906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687de4f-ea8d-49cc-9428-dbb3d1b08036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:36.902515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687de4f-ea8d-49cc-9428-dbb3d1b08036 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.902797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687de4f-ea8d-49cc-9428-dbb3d1b08036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.903069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687de4f-ea8d-49cc-9428-dbb3d1b08036 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.903209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:36.903748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687de4f-ea8d-49cc-9428-dbb3d1b08036 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:36.905371 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f687de4f-ea8d-49cc-9428-dbb3d1b08036 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:36.915777 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1297/2581] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:24:36.920289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687de4f-ea8d-49cc-9428-dbb3d1b08036 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:36.933344 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f687de4f-ea8d-49cc-9428-dbb3d1b08036 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:36.939123 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9326786-ee46-4bac-b058-41025148c72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:36.939793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9326786-ee46-4bac-b058-41025148c72b admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.940067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9326786-ee46-4bac-b058-41025148c72b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.940350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9326786-ee46-4bac-b058-41025148c72b admin admin] PATH_INFO: `/v3/projects/aa84b33c89fa4703b787dc649cc37c9e/users/9a4bc5055ff64ea3b7fb5f3d501d0ade/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.953501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00e107b8a45b4fdfa2cea64ab42180e0'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.954180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00e107b8a45b4fdfa2cea64ab42180e0'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.955689 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9326786-ee46-4bac-b058-41025148c72b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa84b33c89fa4703b787dc649cc37c9e, user_id=9a4bc5055ff64ea3b7fb5f3d501d0ade, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:36.958800 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9326786-ee46-4bac-b058-41025148c72b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:36.963638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00e107b8a45b4fdfa2cea64ab42180e0'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.971218 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00e107b8a45b4fdfa2cea64ab42180e0'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.971907 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-7c044f52-d87a-43ca-b539-e84efd62284b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00e107b8a45b4fdfa2cea64ab42180e0'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:36.973239 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1298/2582] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:36 2022] PUT /identity/v3/projects/aa84b33c89fa4703b787dc649cc37c9e/users/9a4bc5055ff64ea3b7fb5f3d501d0ade/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:36.984791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7c059-0bd8-4f9e-922f-2c51bdcdf0ed None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.985049 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7c059-0bd8-4f9e-922f-2c51bdcdf0ed None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.985318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7c059-0bd8-4f9e-922f-2c51bdcdf0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:36.990294 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1285/2583] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:36 2022] DELETE /identity/v3/users/00e107b8a45b4fdfa2cea64ab42180e0 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:24:36.994527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e1103-6d11-4eba-927c-85053a9e8140 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:36.994814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e1103-6d11-4eba-927c-85053a9e8140 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:36.995095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163e1103-6d11-4eba-927c-85053a9e8140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.014299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-6dd7c059-0bd8-4f9e-922f-2c51bdcdf0ed None None] MFA Rules not processed for user `9a4bc5055ff64ea3b7fb5f3d501d0ade`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:37.015283 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-163e1103-6d11-4eba-927c-85053a9e8140 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:37.015630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6dd7c059-0bd8-4f9e-922f-2c51bdcdf0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:37.016499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-163e1103-6d11-4eba-927c-85053a9e8140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:37.071600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:37.083418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1286/2584] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:24:37.083725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1299/2585] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:36 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:24:37.088075 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e1103-6d11-4eba-927c-85053a9e8140 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.089446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-163e1103-6d11-4eba-927c-85053a9e8140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:37.092532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7c059-0bd8-4f9e-922f-2c51bdcdf0ed None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.093815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6dd7c059-0bd8-4f9e-922f-2c51bdcdf0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:37.101478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163e1103-6d11-4eba-927c-85053a9e8140 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.106124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35921fa4-9309-4872-b7a2-4c2429375ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.106750 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35921fa4-9309-4872-b7a2-4c2429375ae3 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.107023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35921fa4-9309-4872-b7a2-4c2429375ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.107286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35921fa4-9309-4872-b7a2-4c2429375ae3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.107944 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35921fa4-9309-4872-b7a2-4c2429375ae3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.109819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35921fa4-9309-4872-b7a2-4c2429375ae3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.122978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:37.142252 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1287/2586] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:37 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:37.143782 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35921fa4-9309-4872-b7a2-4c2429375ae3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.159040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7c059-0bd8-4f9e-922f-2c51bdcdf0ed None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.212220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eca454-99a4-4145-93a0-f8f581965ea4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.212687 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35921fa4-9309-4872-b7a2-4c2429375ae3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.212830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eca454-99a4-4145-93a0-f8f581965ea4 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.213210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eca454-99a4-4145-93a0-f8f581965ea4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.213482 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eca454-99a4-4145-93a0-f8f581965ea4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.217423 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c725cc-9df3-490c-9122-0a7bfcb041d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.218024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c725cc-9df3-490c-9122-0a7bfcb041d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.218300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c725cc-9df3-490c-9122-0a7bfcb041d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.218570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c725cc-9df3-490c-9122-0a7bfcb041d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.219232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c725cc-9df3-490c-9122-0a7bfcb041d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.221193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c725cc-9df3-490c-9122-0a7bfcb041d6 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.223305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eca454-99a4-4145-93a0-f8f581965ea4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.224875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eca454-99a4-4145-93a0-f8f581965ea4 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.237147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1300/2587] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:24:37.247483 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1288/2588] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:37 2022] POST /identity/v3/users => generated 308 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:24:37.248652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eca454-99a4-4145-93a0-f8f581965ea4 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.258708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eca454-99a4-4145-93a0-f8f581965ea4 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.265165 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9f337-386d-46a7-a251-219cd61cf812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.265821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9f337-386d-46a7-a251-219cd61cf812 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.266088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9f337-386d-46a7-a251-219cd61cf812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.266377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce9f337-386d-46a7-a251-219cd61cf812 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.267008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9f337-386d-46a7-a251-219cd61cf812 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.268664 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce9f337-386d-46a7-a251-219cd61cf812 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.279735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1301/2589] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:37.284253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c725cc-9df3-490c-9122-0a7bfcb041d6 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.295635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c725cc-9df3-490c-9122-0a7bfcb041d6 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.301442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027171e-8b25-4358-af1f-6aa0cb8bb447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.302109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027171e-8b25-4358-af1f-6aa0cb8bb447 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.302389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027171e-8b25-4358-af1f-6aa0cb8bb447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.302667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027171e-8b25-4358-af1f-6aa0cb8bb447 admin admin] PATH_INFO: `/v3/projects/4248671e5d704fdd8d0d54dfb63d4070/users/7ed47dc97ab14401bb16b939a9cc44ff/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.320036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027171e-8b25-4358-af1f-6aa0cb8bb447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4248671e5d704fdd8d0d54dfb63d4070, user_id=7ed47dc97ab14401bb16b939a9cc44ff, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.322718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027171e-8b25-4358-af1f-6aa0cb8bb447 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.336316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1289/2590] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:37 2022] PUT /identity/v3/projects/4248671e5d704fdd8d0d54dfb63d4070/users/7ed47dc97ab14401bb16b939a9cc44ff/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:37.340415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9f337-386d-46a7-a251-219cd61cf812 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.351734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce9f337-386d-46a7-a251-219cd61cf812 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.389960 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027171e-8b25-4358-af1f-6aa0cb8bb447 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.391735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7027171e-8b25-4358-af1f-6aa0cb8bb447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:37.421126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086839c-5237-4da0-a489-f96469b6ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.421735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086839c-5237-4da0-a489-f96469b6ee19 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.422012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086839c-5237-4da0-a489-f96469b6ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.423120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0086839c-5237-4da0-a489-f96469b6ee19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.423869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086839c-5237-4da0-a489-f96469b6ee19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.426063 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0086839c-5237-4da0-a489-f96469b6ee19 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.437319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1302/2591] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:37.442078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086839c-5237-4da0-a489-f96469b6ee19 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.454433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0086839c-5237-4da0-a489-f96469b6ee19 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.459789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22a1ac-e73a-430f-8c30-226cf5c14e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.460512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22a1ac-e73a-430f-8c30-226cf5c14e61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.460855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22a1ac-e73a-430f-8c30-226cf5c14e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.461194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22a1ac-e73a-430f-8c30-226cf5c14e61 admin admin] PATH_INFO: `/v3/projects/4248671e5d704fdd8d0d54dfb63d4070/users/7ed47dc97ab14401bb16b939a9cc44ff/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.463261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22a1ac-e73a-430f-8c30-226cf5c14e61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4248671e5d704fdd8d0d54dfb63d4070, user_id=7ed47dc97ab14401bb16b939a9cc44ff, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.466195 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22a1ac-e73a-430f-8c30-226cf5c14e61 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.478852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1303/2592] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:37 2022] PUT /identity/v3/projects/4248671e5d704fdd8d0d54dfb63d4070/users/7ed47dc97ab14401bb16b939a9cc44ff/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:24:37.489657 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a506d-e6c5-46e4-83d8-4f921331e9c8 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.489933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a506d-e6c5-46e4-83d8-4f921331e9c8 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.490205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a506d-e6c5-46e4-83d8-4f921331e9c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.491163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027171e-8b25-4358-af1f-6aa0cb8bb447 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.520656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-f56a506d-e6c5-46e4-83d8-4f921331e9c8 None None] MFA Rules not processed for user `7ed47dc97ab14401bb16b939a9cc44ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:37.521873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f56a506d-e6c5-46e4-83d8-4f921331e9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:37.549815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e17e9-573d-4040-9850-a3935f588c38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.551259 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e17e9-573d-4040-9850-a3935f588c38 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.551872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e17e9-573d-4040-9850-a3935f588c38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.552221 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e17e9-573d-4040-9850-a3935f588c38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.554969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-564e17e9-573d-4040-9850-a3935f588c38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:37.566434 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:37.579706 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:37.606003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1304/2593] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:37 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:24:37.607015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a506d-e6c5-46e4-83d8-4f921331e9c8 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.608592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-f56a506d-e6c5-46e4-83d8-4f921331e9c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:37.648177 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e17e9-573d-4040-9850-a3935f588c38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.649718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e17e9-573d-4040-9850-a3935f588c38 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.669749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1290/2594] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:24:37.670711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e17e9-573d-4040-9850-a3935f588c38 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.672035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-564e17e9-573d-4040-9850-a3935f588c38 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:37.673355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a506d-e6c5-46e4-83d8-4f921331e9c8 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.679308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.680876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.681178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.681452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] PATH_INFO: `/v3/projects/97e6e815f17446ca846f5e162df7936d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.682959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=97e6e815f17446ca846f5e162df7936d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.684712 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e17e9-573d-4040-9850-a3935f588c38 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.684947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.689173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e02de-c38c-45b8-9652-8765905cea21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.689981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e02de-c38c-45b8-9652-8765905cea21 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.690425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e02de-c38c-45b8-9652-8765905cea21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.690695 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e02de-c38c-45b8-9652-8765905cea21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.701247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e02de-c38c-45b8-9652-8765905cea21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.703770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530e02de-c38c-45b8-9652-8765905cea21 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.717399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1291/2595] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:37 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:37.743172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97e6e815f17446ca846f5e162df7936d', 'request_id': 'req-9f74f02b-9c1e-468f-9464-9a811aca96ee'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:37.750300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97e6e815f17446ca846f5e162df7936d', 'request_id': 'req-9f74f02b-9c1e-468f-9464-9a811aca96ee'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:37.768012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97e6e815f17446ca846f5e162df7936d', 'request_id': 'req-9f74f02b-9c1e-468f-9464-9a811aca96ee'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:37.782886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] The token cache is being invalidate because project 97e6e815f17446ca846f5e162df7936d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:37.783177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:37.783869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:37.786703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1305/2596] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:37 2022] DELETE /identity/v3/projects/97e6e815f17446ca846f5e162df7936d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:37.790785 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e02de-c38c-45b8-9652-8765905cea21 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.810869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530e02de-c38c-45b8-9652-8765905cea21 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.884364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.885025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.885297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.885564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] PATH_INFO: `/v3/users/c0411d53fe85439b8ddf02ca4dc4419f` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.886807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0411d53fe85439b8ddf02ca4dc4419f)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.888788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.901167 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:37.902595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:37.916538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f74f02b-9c1e-468f-9464-9a811aca96ee admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:37.924457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:37.925869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:37.926439 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:37.926857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] PATH_INFO: `/v3/users/9a4bc5055ff64ea3b7fb5f3d501d0ade` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:37.928278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a4bc5055ff64ea3b7fb5f3d501d0ade)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:37.930404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:37.948031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0411d53fe85439b8ddf02ca4dc4419f'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:37.948974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0411d53fe85439b8ddf02ca4dc4419f'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:37.958095 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0411d53fe85439b8ddf02ca4dc4419f'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:37.968300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0411d53fe85439b8ddf02ca4dc4419f'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:37.969465 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a62f0533-1b1f-4954-8f9c-1f9517b26dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0411d53fe85439b8ddf02ca4dc4419f'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:37.978078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:37.991011 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a4bc5055ff64ea3b7fb5f3d501d0ade'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.007507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1292/2597] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:37 2022] DELETE /identity/v3/users/c0411d53fe85439b8ddf02ca4dc4419f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:38.007704 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a4bc5055ff64ea3b7fb5f3d501d0ade'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.008248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a4bc5055ff64ea3b7fb5f3d501d0ade'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.008941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a4bc5055ff64ea3b7fb5f3d501d0ade'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.019993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a4bc5055ff64ea3b7fb5f3d501d0ade'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.022187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:38.044000 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1306/2598] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:37 2022] DELETE /identity/v3/users/9a4bc5055ff64ea3b7fb5f3d501d0ade => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:24:38.048274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612b7f3-257b-4fb2-b068-c3f85bed41c0 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:38.048552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612b7f3-257b-4fb2-b068-c3f85bed41c0 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:38.048835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d612b7f3-257b-4fb2-b068-c3f85bed41c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:38.069682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-d612b7f3-257b-4fb2-b068-c3f85bed41c0 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:38.070936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d612b7f3-257b-4fb2-b068-c3f85bed41c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:38.142391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1293/2599] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:24:38.157871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:38.159900 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:38.240433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c967d6da-67e1-4aa7-989b-b7d346a0a54e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:38.244052 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb16e9-5734-4268-abb7-67751e00bc33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:38.244648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb16e9-5734-4268-abb7-67751e00bc33 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:38.244919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb16e9-5734-4268-abb7-67751e00bc33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:38.245185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb16e9-5734-4268-abb7-67751e00bc33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:38.255773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb16e9-5734-4268-abb7-67751e00bc33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:38.257286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb16e9-5734-4268-abb7-67751e00bc33 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:38.266828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1307/2600] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:38.328945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612b7f3-257b-4fb2-b068-c3f85bed41c0 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:38.330676 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d612b7f3-257b-4fb2-b068-c3f85bed41c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:38.343683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d612b7f3-257b-4fb2-b068-c3f85bed41c0 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:38.350411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:38.351550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:38.351847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:38.352291 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] PATH_INFO: `/v3/projects/13cf0f607293448c9bf2817d953aa18c` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:38.353558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13cf0f607293448c9bf2817d953aa18c)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:38.358057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:38.414577 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13cf0f607293448c9bf2817d953aa18c', 'request_id': 'req-97844361-64af-48de-9c6c-17d80cb23563'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.422534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13cf0f607293448c9bf2817d953aa18c', 'request_id': 'req-97844361-64af-48de-9c6c-17d80cb23563'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.430638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13cf0f607293448c9bf2817d953aa18c', 'request_id': 'req-97844361-64af-48de-9c6c-17d80cb23563'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.446238 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] The token cache is being invalidate because project 13cf0f607293448c9bf2817d953aa18c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:38.446655 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.447306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.448857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1294/2601] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:38 2022] DELETE /identity/v3/projects/13cf0f607293448c9bf2817d953aa18c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:38.452722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb16e9-5734-4268-abb7-67751e00bc33 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:38.463556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb16e9-5734-4268-abb7-67751e00bc33 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:38.526514 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:38.527139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:38.527413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:38.527714 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] PATH_INFO: `/v3/users/646b64a19dc641f094b0e84357296faf` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:38.528825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] RBAC: Authorizing `identity:delete_user(user_id=646b64a19dc641f094b0e84357296faf)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:38.531755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:38.583887 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '646b64a19dc641f094b0e84357296faf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.591677 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '646b64a19dc641f094b0e84357296faf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.592373 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '646b64a19dc641f094b0e84357296faf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.593170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '646b64a19dc641f094b0e84357296faf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.613530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '646b64a19dc641f094b0e84357296faf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.623143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:38.625066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:38.647170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1308/2602] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:38 2022] DELETE /identity/v3/users/646b64a19dc641f094b0e84357296faf => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:38.649855 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97844361-64af-48de-9c6c-17d80cb23563 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:38.719583 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:38.720310 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:38.720652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:38.721007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] PATH_INFO: `/v3/projects/aa84b33c89fa4703b787dc649cc37c9e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:38.723272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa84b33c89fa4703b787dc649cc37c9e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:38.725557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:38.742682 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:38.776402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa84b33c89fa4703b787dc649cc37c9e', 'request_id': 'req-214559d0-539e-4662-bf49-9f22ce04a20a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.787103 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa84b33c89fa4703b787dc649cc37c9e', 'request_id': 'req-214559d0-539e-4662-bf49-9f22ce04a20a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.792913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa84b33c89fa4703b787dc649cc37c9e', 'request_id': 'req-214559d0-539e-4662-bf49-9f22ce04a20a'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.805637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] The token cache is being invalidate because project aa84b33c89fa4703b787dc649cc37c9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:38.805933 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.806607 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.808328 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1295/2603] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:38 2022] DELETE /identity/v3/projects/aa84b33c89fa4703b787dc649cc37c9e => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:24:38.813456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:38.816260 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:38.825921 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:38.830245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d5e195-1104-44e8-80d3-8dc0a8311175 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:38.893003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:38.893630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:38.893913 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:38.894214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] PATH_INFO: `/v3/users/7ed47dc97ab14401bb16b939a9cc44ff` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:38.895720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed47dc97ab14401bb16b939a9cc44ff)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:38.897648 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:38.934325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:38.936099 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:38.951496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214559d0-539e-4662-bf49-9f22ce04a20a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:38.956231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:38.957535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:38.958069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:38.958346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] PATH_INFO: `/v3/projects/4e2e4361992b49959c4426d9d5aa9670` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:38.959560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e2e4361992b49959c4426d9d5aa9670)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:38.961291 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed47dc97ab14401bb16b939a9cc44ff'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.961400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:38.972993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed47dc97ab14401bb16b939a9cc44ff'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.973928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed47dc97ab14401bb16b939a9cc44ff'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.975001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed47dc97ab14401bb16b939a9cc44ff'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:38.983433 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-369cf12e-1a90-47f9-a6f7-4b6aca1a826b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed47dc97ab14401bb16b939a9cc44ff'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.002852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1309/2604] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:38 2022] DELETE /identity/v3/users/7ed47dc97ab14401bb16b939a9cc44ff => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:39.014910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e2e4361992b49959c4426d9d5aa9670', 'request_id': 'req-bff84e7e-2978-48a2-9079-91e7169c0fa1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.023667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e2e4361992b49959c4426d9d5aa9670', 'request_id': 'req-bff84e7e-2978-48a2-9079-91e7169c0fa1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.029415 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e2e4361992b49959c4426d9d5aa9670', 'request_id': 'req-bff84e7e-2978-48a2-9079-91e7169c0fa1'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.041236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] The token cache is being invalidate because project 4e2e4361992b49959c4426d9d5aa9670 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:39.041524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.042181 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bff84e7e-2978-48a2-9079-91e7169c0fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.043763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1296/2605] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:38 2022] DELETE /identity/v3/projects/4e2e4361992b49959c4426d9d5aa9670 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:39.049994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28250b98-245c-42a8-8674-f71208a728ff None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.050357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28250b98-245c-42a8-8674-f71208a728ff None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.050635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28250b98-245c-42a8-8674-f71208a728ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.071177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-28250b98-245c-42a8-8674-f71208a728ff None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:39.072529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-28250b98-245c-42a8-8674-f71208a728ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:39.137756 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1310/2606] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:39.147375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73635f-6efe-4a9f-88c9-fee5c79f051e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.147886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73635f-6efe-4a9f-88c9-fee5c79f051e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.147886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73635f-6efe-4a9f-88c9-fee5c79f051e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.169436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-af73635f-6efe-4a9f-88c9-fee5c79f051e None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:39.170729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-af73635f-6efe-4a9f-88c9-fee5c79f051e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:39.179343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1297/2607] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:39.183641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28250b98-245c-42a8-8674-f71208a728ff None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.193994 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28250b98-245c-42a8-8674-f71208a728ff None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.198709 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56506e1a-f4bb-49c1-b760-ed5694ff8dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.199373 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56506e1a-f4bb-49c1-b760-ed5694ff8dc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.199703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56506e1a-f4bb-49c1-b760-ed5694ff8dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.199980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56506e1a-f4bb-49c1-b760-ed5694ff8dc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.200625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56506e1a-f4bb-49c1-b760-ed5694ff8dc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:24:39.200934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56506e1a-f4bb-49c1-b760-ed5694ff8dc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.202730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56506e1a-f4bb-49c1-b760-ed5694ff8dc5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.216412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1311/2608] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:24:39.220671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73635f-6efe-4a9f-88c9-fee5c79f051e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.232129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73635f-6efe-4a9f-88c9-fee5c79f051e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.240438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b2c7b-44ca-40c0-ad26-40cad0f8a5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.241654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b2c7b-44ca-40c0-ad26-40cad0f8a5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.242020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b2c7b-44ca-40c0-ad26-40cad0f8a5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.242389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9b2c7b-44ca-40c0-ad26-40cad0f8a5c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.243161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b2c7b-44ca-40c0-ad26-40cad0f8a5c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.245241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9b2c7b-44ca-40c0-ad26-40cad0f8a5c3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.274356 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1298/2609] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:39 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:24:39.279294 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56506e1a-f4bb-49c1-b760-ed5694ff8dc5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.290559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56506e1a-f4bb-49c1-b760-ed5694ff8dc5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.298533 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:39.347329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b2c7b-44ca-40c0-ad26-40cad0f8a5c3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.349163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6c9b2c7b-44ca-40c0-ad26-40cad0f8a5c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:39.369108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179e72a-7d04-4988-a434-5a6a7eebf0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.369731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179e72a-7d04-4988-a434-5a6a7eebf0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.369980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179e72a-7d04-4988-a434-5a6a7eebf0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.370246 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9179e72a-7d04-4988-a434-5a6a7eebf0f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.371030 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179e72a-7d04-4988-a434-5a6a7eebf0f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.372858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9179e72a-7d04-4988-a434-5a6a7eebf0f8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.405562 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1312/2610] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:39 2022] POST /identity/v3/users => generated 284 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:39.407216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179e72a-7d04-4988-a434-5a6a7eebf0f8 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.420688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9179e72a-7d04-4988-a434-5a6a7eebf0f8 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.431505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926dcb7-af3c-48a0-ba4c-fc2d3114edd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.432113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926dcb7-af3c-48a0-ba4c-fc2d3114edd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.432417 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926dcb7-af3c-48a0-ba4c-fc2d3114edd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.432722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926dcb7-af3c-48a0-ba4c-fc2d3114edd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.433367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926dcb7-af3c-48a0-ba4c-fc2d3114edd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.435334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926dcb7-af3c-48a0-ba4c-fc2d3114edd4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.437441 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:39.442608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9b2c7b-44ca-40c0-ad26-40cad0f8a5c3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.448746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.449729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.450296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1313/2611] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:39.450505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.451080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] PATH_INFO: `/v3/projects/4248671e5d704fdd8d0d54dfb63d4070` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.452254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4248671e5d704fdd8d0d54dfb63d4070)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.454169 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.454293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926dcb7-af3c-48a0-ba4c-fc2d3114edd4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.464872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926dcb7-af3c-48a0-ba4c-fc2d3114edd4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.469366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e5694-0306-446d-b2f2-a9cf9eb5bde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.469938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e5694-0306-446d-b2f2-a9cf9eb5bde0 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.470207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e5694-0306-446d-b2f2-a9cf9eb5bde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.470477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e5694-0306-446d-b2f2-a9cf9eb5bde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.471098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e5694-0306-446d-b2f2-a9cf9eb5bde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.473237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e5694-0306-446d-b2f2-a9cf9eb5bde0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.483354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1314/2612] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:39.488264 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e5694-0306-446d-b2f2-a9cf9eb5bde0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.498535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e5694-0306-446d-b2f2-a9cf9eb5bde0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.502151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4248671e5d704fdd8d0d54dfb63d4070', 'request_id': 'req-791bc0d8-490d-4e30-8f82-029164155712'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.510523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4248671e5d704fdd8d0d54dfb63d4070', 'request_id': 'req-791bc0d8-490d-4e30-8f82-029164155712'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.516360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4248671e5d704fdd8d0d54dfb63d4070', 'request_id': 'req-791bc0d8-490d-4e30-8f82-029164155712'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.527375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] The token cache is being invalidate because project 4248671e5d704fdd8d0d54dfb63d4070 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:39.527692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.528349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-791bc0d8-490d-4e30-8f82-029164155712 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:39.529838 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1299/2613] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:39 2022] DELETE /identity/v3/projects/4248671e5d704fdd8d0d54dfb63d4070 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:39.534783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d8c94-e60f-4fa8-8889-3e8f13e509d8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.535123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d8c94-e60f-4fa8-8889-3e8f13e509d8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.535401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d8c94-e60f-4fa8-8889-3e8f13e509d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.555493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-8d7d8c94-e60f-4fa8-8889-3e8f13e509d8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:39.556647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-8d7d8c94-e60f-4fa8-8889-3e8f13e509d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:39.560640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca718b3f-fb11-4913-a47d-a042db022c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.561297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca718b3f-fb11-4913-a47d-a042db022c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.561623 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca718b3f-fb11-4913-a47d-a042db022c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.561898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca718b3f-fb11-4913-a47d-a042db022c5b admin admin] PATH_INFO: `/v3/projects/599ffb54b5994e428b84adfd9c38481b/users/7dcf25bcabb342cf895333fce7e422ef/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.564869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1300/2614] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:24:39.573269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac86b0-b050-498d-b854-8df5819e197c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.573548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac86b0-b050-498d-b854-8df5819e197c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.573817 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdac86b0-b050-498d-b854-8df5819e197c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.579683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca718b3f-fb11-4913-a47d-a042db022c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=599ffb54b5994e428b84adfd9c38481b, user_id=7dcf25bcabb342cf895333fce7e422ef, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.583177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca718b3f-fb11-4913-a47d-a042db022c5b admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.594889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-bdac86b0-b050-498d-b854-8df5819e197c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:39.596233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bdac86b0-b050-498d-b854-8df5819e197c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:39.596498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1315/2615] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:39 2022] PUT /identity/v3/projects/599ffb54b5994e428b84adfd9c38481b/users/7dcf25bcabb342cf895333fce7e422ef/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:24:39.607125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84627c-4df2-4eac-b0c5-bbe36a0a9804 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.607406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84627c-4df2-4eac-b0c5-bbe36a0a9804 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.607743 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84627c-4df2-4eac-b0c5-bbe36a0a9804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.636204 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2a84627c-4df2-4eac-b0c5-bbe36a0a9804 None None] MFA Rules not processed for user `7dcf25bcabb342cf895333fce7e422ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:39.637473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2a84627c-4df2-4eac-b0c5-bbe36a0a9804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:39.662359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1301/2616] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:39.666529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac86b0-b050-498d-b854-8df5819e197c None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.675745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdac86b0-b050-498d-b854-8df5819e197c None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.680690 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93423f-4dbf-4261-956f-c2988e0f455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.681279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93423f-4dbf-4261-956f-c2988e0f455c admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.681550 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93423f-4dbf-4261-956f-c2988e0f455c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.681822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93423f-4dbf-4261-956f-c2988e0f455c admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.682459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93423f-4dbf-4261-956f-c2988e0f455c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:24:39.682753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93423f-4dbf-4261-956f-c2988e0f455c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.684373 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93423f-4dbf-4261-956f-c2988e0f455c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.688030 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:39.697053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1302/2617] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:24:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:24:39.701260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93423f-4dbf-4261-956f-c2988e0f455c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.707902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1316/2618] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:24:39.711488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93423f-4dbf-4261-956f-c2988e0f455c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.716470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c469a-020b-4524-bf73-ea9c81cd4521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.717063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c469a-020b-4524-bf73-ea9c81cd4521 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.717338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c469a-020b-4524-bf73-ea9c81cd4521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.717606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c469a-020b-4524-bf73-ea9c81cd4521 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.718224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c469a-020b-4524-bf73-ea9c81cd4521 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.720157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c469a-020b-4524-bf73-ea9c81cd4521 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.721404 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84627c-4df2-4eac-b0c5-bbe36a0a9804 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.723296 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2a84627c-4df2-4eac-b0c5-bbe36a0a9804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:39.755158 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1303/2619] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:39 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:24:39.756643 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c469a-020b-4524-bf73-ea9c81cd4521 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.781860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84627c-4df2-4eac-b0c5-bbe36a0a9804 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.806555 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:39.823408 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c469a-020b-4524-bf73-ea9c81cd4521 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.828102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8691aa-b3d3-45ae-83d5-c45408430c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.828707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8691aa-b3d3-45ae-83d5-c45408430c69 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.828980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8691aa-b3d3-45ae-83d5-c45408430c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.829242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8691aa-b3d3-45ae-83d5-c45408430c69 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.829925 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8691aa-b3d3-45ae-83d5-c45408430c69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.832020 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8691aa-b3d3-45ae-83d5-c45408430c69 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.832566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb828cf6-45bb-4319-b4b6-4b28f3a3d35a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf75b583add44b6b87bff5dc67080d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.833175 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb828cf6-45bb-4319-b4b6-4b28f3a3d35a service glance] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.833452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb828cf6-45bb-4319-b4b6-4b28f3a3d35a service glance] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.833722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb828cf6-45bb-4319-b4b6-4b28f3a3d35a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.857826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1304/2620] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:39 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:24:39.859101 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8691aa-b3d3-45ae-83d5-c45408430c69 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.870672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8691aa-b3d3-45ae-83d5-c45408430c69 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.875702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6555cc7-eac6-4528-8843-40bba4587186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.876288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6555cc7-eac6-4528-8843-40bba4587186 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.876560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6555cc7-eac6-4528-8843-40bba4587186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.876829 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6555cc7-eac6-4528-8843-40bba4587186 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.877473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6555cc7-eac6-4528-8843-40bba4587186 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.879950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6555cc7-eac6-4528-8843-40bba4587186 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.890308 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1305/2621] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:24:39.891765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb828cf6-45bb-4319-b4b6-4b28f3a3d35a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.893487 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb828cf6-45bb-4319-b4b6-4b28f3a3d35a service glance] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.894941 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6555cc7-eac6-4528-8843-40bba4587186 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.903852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1317/2622] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:24:39.904988 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6555cc7-eac6-4528-8843-40bba4587186 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.909631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7e8e3-c9c6-49bf-b7ab-45b7384b855e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.910564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e8e3-c9c6-49bf-b7ab-45b7384b855e admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.910843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e8e3-c9c6-49bf-b7ab-45b7384b855e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.911109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7e8e3-c9c6-49bf-b7ab-45b7384b855e admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.911791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7e8e3-c9c6-49bf-b7ab-45b7384b855e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.913377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7e8e3-c9c6-49bf-b7ab-45b7384b855e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.926504 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1306/2623] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:24:39.932840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb828cf6-45bb-4319-b4b6-4b28f3a3d35a service glance] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:39.945770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb828cf6-45bb-4319-b4b6-4b28f3a3d35a service glance] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:39.951504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49425bf3-4c53-4efc-a6c8-3695e512694f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:39.952192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49425bf3-4c53-4efc-a6c8-3695e512694f admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:39.952467 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49425bf3-4c53-4efc-a6c8-3695e512694f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:39.952857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49425bf3-4c53-4efc-a6c8-3695e512694f admin admin] PATH_INFO: `/v3/projects/49973800fe3b4cccbf9ee95361330830/users/1c100121ca05424ebf9f470b89f8acbb/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:39.970565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49425bf3-4c53-4efc-a6c8-3695e512694f admin admin] RBAC: Authorizing `identity:create_grant(project_id=49973800fe3b4cccbf9ee95361330830, user_id=1c100121ca05424ebf9f470b89f8acbb, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:39.974669 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49425bf3-4c53-4efc-a6c8-3695e512694f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:39.994019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1318/2624] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:39 2022] PUT /identity/v3/projects/49973800fe3b4cccbf9ee95361330830/users/1c100121ca05424ebf9f470b89f8acbb/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:40.005822 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b9b83-e0e8-4d11-acc9-6f00e7bed230 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:40.006110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b9b83-e0e8-4d11-acc9-6f00e7bed230 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:40.006385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b9b83-e0e8-4d11-acc9-6f00e7bed230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:40.039155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-f57b9b83-e0e8-4d11-acc9-6f00e7bed230 None None] MFA Rules not processed for user `1c100121ca05424ebf9f470b89f8acbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:40.040414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f57b9b83-e0e8-4d11-acc9-6f00e7bed230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:40.115872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1307/2625] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:40.120369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49425bf3-4c53-4efc-a6c8-3695e512694f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:40.168515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:40.196154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49425bf3-4c53-4efc-a6c8-3695e512694f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:40.202242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27cbbd-c8eb-4799-83ee-f7ff39ee0fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:40.203573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27cbbd-c8eb-4799-83ee-f7ff39ee0fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:40.203919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27cbbd-c8eb-4799-83ee-f7ff39ee0fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:40.204257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f27cbbd-c8eb-4799-83ee-f7ff39ee0fc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:40.205322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27cbbd-c8eb-4799-83ee-f7ff39ee0fc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:40.208106 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f27cbbd-c8eb-4799-83ee-f7ff39ee0fc4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:40.247244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1319/2626] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:40 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:40.249121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b9b83-e0e8-4d11-acc9-6f00e7bed230 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:40.264611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b9b83-e0e8-4d11-acc9-6f00e7bed230 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:40.336862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb49ee-3edc-4daf-827a-68be7b6f691a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:40.337497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb49ee-3edc-4daf-827a-68be7b6f691a admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:40.338340 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb49ee-3edc-4daf-827a-68be7b6f691a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:40.338611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb49ee-3edc-4daf-827a-68be7b6f691a admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:40.339254 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb49ee-3edc-4daf-827a-68be7b6f691a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:40.341370 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddb49ee-3edc-4daf-827a-68be7b6f691a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:40.370782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27cbbd-c8eb-4799-83ee-f7ff39ee0fc4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:40.379849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1308/2627] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:40 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:24:40.385237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f27cbbd-c8eb-4799-83ee-f7ff39ee0fc4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:40.392028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ed28d-9896-4c7c-a9ce-80676b15cb9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:40.393041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ed28d-9896-4c7c-a9ce-80676b15cb9e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:40.393318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ed28d-9896-4c7c-a9ce-80676b15cb9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:40.394569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0ed28d-9896-4c7c-a9ce-80676b15cb9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:40.396699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ed28d-9896-4c7c-a9ce-80676b15cb9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:40.398545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0ed28d-9896-4c7c-a9ce-80676b15cb9e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:40.412456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1320/2628] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:40.416612 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb49ee-3edc-4daf-827a-68be7b6f691a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:40.445452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddb49ee-3edc-4daf-827a-68be7b6f691a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:40.456184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec62e18-9265-4e06-b4ca-7362fdba4178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:40.458891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec62e18-9265-4e06-b4ca-7362fdba4178 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:40.459205 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec62e18-9265-4e06-b4ca-7362fdba4178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:40.459733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec62e18-9265-4e06-b4ca-7362fdba4178 admin admin] PATH_INFO: `/v3/projects/39b34c6327d04518aecadc8dae713270/users/531c382338974251ab773eaf9d9bad8a/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:40.482299 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec62e18-9265-4e06-b4ca-7362fdba4178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b34c6327d04518aecadc8dae713270, user_id=531c382338974251ab773eaf9d9bad8a, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:40.484832 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec62e18-9265-4e06-b4ca-7362fdba4178 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:40.502635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1309/2629] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:40 2022] PUT /identity/v3/projects/39b34c6327d04518aecadc8dae713270/users/531c382338974251ab773eaf9d9bad8a/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:40.507042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ed28d-9896-4c7c-a9ce-80676b15cb9e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:40.517290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0ed28d-9896-4c7c-a9ce-80676b15cb9e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:40.581116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0504fa8-1728-4ac2-bed7-dd7bc3018a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:40.582408 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0504fa8-1728-4ac2-bed7-dd7bc3018a4a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:40.582683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0504fa8-1728-4ac2-bed7-dd7bc3018a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:40.582948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0504fa8-1728-4ac2-bed7-dd7bc3018a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:40.583621 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0504fa8-1728-4ac2-bed7-dd7bc3018a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:40.585369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0504fa8-1728-4ac2-bed7-dd7bc3018a4a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:40.595631 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1321/2630] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:40.599992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec62e18-9265-4e06-b4ca-7362fdba4178 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:40.610739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec62e18-9265-4e06-b4ca-7362fdba4178 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:40.617217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b301981-907f-46b2-bbe3-cd32a2e59f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:40.617883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b301981-907f-46b2-bbe3-cd32a2e59f10 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:40.618159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b301981-907f-46b2-bbe3-cd32a2e59f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:40.618429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b301981-907f-46b2-bbe3-cd32a2e59f10 admin admin] PATH_INFO: `/v3/projects/39b34c6327d04518aecadc8dae713270/users/531c382338974251ab773eaf9d9bad8a/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:40.620501 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b301981-907f-46b2-bbe3-cd32a2e59f10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=39b34c6327d04518aecadc8dae713270, user_id=531c382338974251ab773eaf9d9bad8a, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:40.624072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b301981-907f-46b2-bbe3-cd32a2e59f10 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:40.627843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:40.639610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1310/2631] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:40 2022] PUT /identity/v3/projects/39b34c6327d04518aecadc8dae713270/users/531c382338974251ab773eaf9d9bad8a/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:40.650937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed3e39-fda4-4ced-9e4d-fbe172ed1d26 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:40.652043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed3e39-fda4-4ced-9e4d-fbe172ed1d26 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:40.652401 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed3e39-fda4-4ced-9e4d-fbe172ed1d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:40.714131 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2fed3e39-fda4-4ced-9e4d-fbe172ed1d26 None None] MFA Rules not processed for user `531c382338974251ab773eaf9d9bad8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:40.715351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2fed3e39-fda4-4ced-9e4d-fbe172ed1d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:40.801369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1322/2632] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:40.808884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b301981-907f-46b2-bbe3-cd32a2e59f10 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:40.811192 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-2b301981-907f-46b2-bbe3-cd32a2e59f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:40.877768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b301981-907f-46b2-bbe3-cd32a2e59f10 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:40.883290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e621069-05f8-40ad-a7b4-6a1e19aa471a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:40.884208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e621069-05f8-40ad-a7b4-6a1e19aa471a service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:40.884601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e621069-05f8-40ad-a7b4-6a1e19aa471a service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:40.884913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e621069-05f8-40ad-a7b4-6a1e19aa471a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:40.896293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e621069-05f8-40ad-a7b4-6a1e19aa471a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:40.897902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e621069-05f8-40ad-a7b4-6a1e19aa471a service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:40.910435 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1311/2633] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:24:41.043164 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed3e39-fda4-4ced-9e4d-fbe172ed1d26 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:41.058305 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed3e39-fda4-4ced-9e4d-fbe172ed1d26 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:41.063891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48e560-b606-448d-bddb-c915f41869f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:41.064659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48e560-b606-448d-bddb-c915f41869f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:41.065053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48e560-b606-448d-bddb-c915f41869f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:41.065424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca48e560-b606-448d-bddb-c915f41869f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:41.120163 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:41.170139 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48e560-b606-448d-bddb-c915f41869f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:41.171863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca48e560-b606-448d-bddb-c915f41869f9 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:41.185952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1323/2634] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:48.895635 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e621069-05f8-40ad-a7b4-6a1e19aa471a service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:48.897325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9e621069-05f8-40ad-a7b4-6a1e19aa471a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:48.956730 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e621069-05f8-40ad-a7b4-6a1e19aa471a service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:48.960394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e64c7-f701-4029-a4c4-69b05eed1186 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:48.960992 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e64c7-f701-4029-a4c4-69b05eed1186 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:48.961265 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e64c7-f701-4029-a4c4-69b05eed1186 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:48.961533 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e64c7-f701-4029-a4c4-69b05eed1186 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:48.963361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d61e64c7-f701-4029-a4c4-69b05eed1186 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:48.974121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e64c7-f701-4029-a4c4-69b05eed1186 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:48.975637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61e64c7-f701-4029-a4c4-69b05eed1186 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:48.985049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1312/2635] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:50.258636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48e560-b606-448d-bddb-c915f41869f9 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:50.261040 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ca48e560-b606-448d-bddb-c915f41869f9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:50.274640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca48e560-b606-448d-bddb-c915f41869f9 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:50.279788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f31f1-d758-49c0-8743-cc2a53dad49e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:50.280413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f31f1-d758-49c0-8743-cc2a53dad49e service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:50.280692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f31f1-d758-49c0-8743-cc2a53dad49e service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:50.280969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f31f1-d758-49c0-8743-cc2a53dad49e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:50.282725 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cb3f31f1-d758-49c0-8743-cc2a53dad49e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:50.356097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f31f1-d758-49c0-8743-cc2a53dad49e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:50.357544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f31f1-d758-49c0-8743-cc2a53dad49e service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:50.367686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1324/2636] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:24:53.544122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e64c7-f701-4029-a4c4-69b05eed1186 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:53.548380 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-d61e64c7-f701-4029-a4c4-69b05eed1186 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:53.566594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e64c7-f701-4029-a4c4-69b05eed1186 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:53.571802 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:53.572566 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:53.573034 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:53.573330 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] PATH_INFO: `/v3/users/810f0fa1f7774b37a09a91964c40b823` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:53.574666 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] RBAC: Authorizing `identity:delete_user(user_id=810f0fa1f7774b37a09a91964c40b823)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:53.576656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:53.647864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '810f0fa1f7774b37a09a91964c40b823'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:53.648622 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '810f0fa1f7774b37a09a91964c40b823'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:53.658374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '810f0fa1f7774b37a09a91964c40b823'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:53.666508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '810f0fa1f7774b37a09a91964c40b823'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:53.667599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bb7e15c2-bdbd-48c7-b556-5a316515f05d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '810f0fa1f7774b37a09a91964c40b823'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:53.695493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1313/2637] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:24:53 2022] DELETE /identity/v3/users/810f0fa1f7774b37a09a91964c40b823 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:54.219991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f31f1-d758-49c0-8743-cc2a53dad49e service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:54.241228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f31f1-d758-49c0-8743-cc2a53dad49e service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:54.307392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:54.308058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:54.308331 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:54.308599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] PATH_INFO: `/v3/projects/7d5947e3bd184558bc1a8febcedce00f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:54.309732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d5947e3bd184558bc1a8febcedce00f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:54.311691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:54.371412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d5947e3bd184558bc1a8febcedce00f', 'request_id': 'req-c9783d6d-d09c-47f7-9d70-5c3189d88db0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:54.377888 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d5947e3bd184558bc1a8febcedce00f', 'request_id': 'req-c9783d6d-d09c-47f7-9d70-5c3189d88db0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:54.386832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d5947e3bd184558bc1a8febcedce00f', 'request_id': 'req-c9783d6d-d09c-47f7-9d70-5c3189d88db0'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:54.389791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:54.403916 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] The token cache is being invalidate because project 7d5947e3bd184558bc1a8febcedce00f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:24:54.404628 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:54.406130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c9783d6d-d09c-47f7-9d70-5c3189d88db0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:24:54.407559 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1325/2638] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:24:54 2022] DELETE /identity/v3/projects/7d5947e3bd184558bc1a8febcedce00f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:24:54.413718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb47889-d2ff-42d1-83b4-16f43bc18c45 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:54.414002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb47889-d2ff-42d1-83b4-16f43bc18c45 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:54.414275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb47889-d2ff-42d1-83b4-16f43bc18c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:54.437137 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7eb47889-d2ff-42d1-83b4-16f43bc18c45 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:54.438385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7eb47889-d2ff-42d1-83b4-16f43bc18c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:54.447284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:24:54.504124 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1314/2639] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:54.511869 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3193-56e8-45fb-87fa-042e22eae75f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:54.512159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3193-56e8-45fb-87fa-042e22eae75f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:54.512435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f3193-56e8-45fb-87fa-042e22eae75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:54.533497 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b19f3193-56e8-45fb-87fa-042e22eae75f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:54.534706 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b19f3193-56e8-45fb-87fa-042e22eae75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:54.543988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1326/2640] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:54.548348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb47889-d2ff-42d1-83b4-16f43bc18c45 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:54.558366 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb47889-d2ff-42d1-83b4-16f43bc18c45 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:54.563011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fce5f-8957-4623-8bf7-f047cdecb2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:54.563678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fce5f-8957-4623-8bf7-f047cdecb2cb admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:54.563954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fce5f-8957-4623-8bf7-f047cdecb2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:54.564235 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fce5f-8957-4623-8bf7-f047cdecb2cb admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:54.564889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fce5f-8957-4623-8bf7-f047cdecb2cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:24:54.565199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fce5f-8957-4623-8bf7-f047cdecb2cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:54.566840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fce5f-8957-4623-8bf7-f047cdecb2cb admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:54.578759 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1315/2641] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:24:54.584028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3193-56e8-45fb-87fa-042e22eae75f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:54.593749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f3193-56e8-45fb-87fa-042e22eae75f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:54.599422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a906f0-5c04-48e4-aae4-03ade8e1d83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:54.600058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a906f0-5c04-48e4-aae4-03ade8e1d83e admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:54.600334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a906f0-5c04-48e4-aae4-03ade8e1d83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:54.600607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a906f0-5c04-48e4-aae4-03ade8e1d83e admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:54.601239 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a906f0-5c04-48e4-aae4-03ade8e1d83e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:54.603160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a906f0-5c04-48e4-aae4-03ade8e1d83e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:54.633813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1327/2642] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:24:54 2022] POST /identity/v3/projects => generated 367 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:24:54.637338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fce5f-8957-4623-8bf7-f047cdecb2cb admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:54.647236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fce5f-8957-4623-8bf7-f047cdecb2cb admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:54.709547 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842742bc-0165-4a4a-a28f-bc8b5e45a2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:54.710194 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842742bc-0165-4a4a-a28f-bc8b5e45a2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:54.710471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842742bc-0165-4a4a-a28f-bc8b5e45a2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:54.710743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842742bc-0165-4a4a-a28f-bc8b5e45a2f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:54.711390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842742bc-0165-4a4a-a28f-bc8b5e45a2f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:54.713385 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842742bc-0165-4a4a-a28f-bc8b5e45a2f2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:54.739581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1316/2643] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:24:54 2022] POST /identity/v3/users => generated 284 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:24:54.743989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a906f0-5c04-48e4-aae4-03ade8e1d83e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:54.754485 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a906f0-5c04-48e4-aae4-03ade8e1d83e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:54.759426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32687001-00c3-4e0d-8b86-1f4d33c14094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:54.760066 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32687001-00c3-4e0d-8b86-1f4d33c14094 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:54.760335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32687001-00c3-4e0d-8b86-1f4d33c14094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:54.760610 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32687001-00c3-4e0d-8b86-1f4d33c14094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:54.761258 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32687001-00c3-4e0d-8b86-1f4d33c14094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:54.762917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32687001-00c3-4e0d-8b86-1f4d33c14094 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:54.773781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1328/2644] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:54.778986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842742bc-0165-4a4a-a28f-bc8b5e45a2f2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:54.789686 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842742bc-0165-4a4a-a28f-bc8b5e45a2f2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:54.794449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8566e9-0723-47ec-b988-fb48a680cf68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:54.795055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8566e9-0723-47ec-b988-fb48a680cf68 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:54.795334 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8566e9-0723-47ec-b988-fb48a680cf68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:54.795644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8566e9-0723-47ec-b988-fb48a680cf68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:54.796288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8566e9-0723-47ec-b988-fb48a680cf68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:54.797897 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8566e9-0723-47ec-b988-fb48a680cf68 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:54.809318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1317/2645] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:24:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:24:54.814067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32687001-00c3-4e0d-8b86-1f4d33c14094 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:54.824716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32687001-00c3-4e0d-8b86-1f4d33c14094 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:54.829511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192a728-ab87-4e28-8efd-eeada10f7b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:54.830161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a728-ab87-4e28-8efd-eeada10f7b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:54.830436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a728-ab87-4e28-8efd-eeada10f7b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:54.830702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6192a728-ab87-4e28-8efd-eeada10f7b09 admin admin] PATH_INFO: `/v3/projects/6915b8ba96034df0811d3a850b5face0/users/ec42ceb381f4453499616ebeb08ebf64/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:54.856606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192a728-ab87-4e28-8efd-eeada10f7b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6915b8ba96034df0811d3a850b5face0, user_id=ec42ceb381f4453499616ebeb08ebf64, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:54.862143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6192a728-ab87-4e28-8efd-eeada10f7b09 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:54.878975 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1329/2646] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:24:54 2022] PUT /identity/v3/projects/6915b8ba96034df0811d3a850b5face0/users/ec42ceb381f4453499616ebeb08ebf64/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:24:54.890313 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdc540-497b-4c36-b953-ef999dd85481 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:54.890636 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdc540-497b-4c36-b953-ef999dd85481 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:54.890806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bdc540-497b-4c36-b953-ef999dd85481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:54.926880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-f0bdc540-497b-4c36-b953-ef999dd85481 None None] MFA Rules not processed for user `ec42ceb381f4453499616ebeb08ebf64`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:24:54.928159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f0bdc540-497b-4c36-b953-ef999dd85481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:54.996765 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1318/2647] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:24:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:24:55.008415 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192a728-ab87-4e28-8efd-eeada10f7b09 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:24:55.010297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6192a728-ab87-4e28-8efd-eeada10f7b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:24:55.079612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6192a728-ab87-4e28-8efd-eeada10f7b09 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:24:55.083948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25e952-374f-482b-aad4-c8ce2758f241 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:24:55.084565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25e952-374f-482b-aad4-c8ce2758f241 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:24:55.084858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25e952-374f-482b-aad4-c8ce2758f241 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:24:55.085118 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25e952-374f-482b-aad4-c8ce2758f241 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:24:55.094699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25e952-374f-482b-aad4-c8ce2758f241 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:24:55.096224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25e952-374f-482b-aad4-c8ce2758f241 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:24:55.107115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1330/2648] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:24:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:04.993551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=93,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:05.102313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=55,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:05.692391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdc540-497b-4c36-b953-ef999dd85481 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:05.753391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bdc540-497b-4c36-b953-ef999dd85481 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:05.758109 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:05.758724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:05.758997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:05.759264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] PATH_INFO: `/v3/users/ec42ceb381f4453499616ebeb08ebf64` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:05.760389 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec42ceb381f4453499616ebeb08ebf64)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:05.762304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:05.808467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec42ceb381f4453499616ebeb08ebf64'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:05.810025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec42ceb381f4453499616ebeb08ebf64'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:05.817439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec42ceb381f4453499616ebeb08ebf64'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:05.825444 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec42ceb381f4453499616ebeb08ebf64'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:05.826115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec42ceb381f4453499616ebeb08ebf64'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:05.845928 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1319/2649] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:05 2022] DELETE /identity/v3/users/ec42ceb381f4453499616ebeb08ebf64 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:05.851138 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2dfa7-2a1a-4b2e-b7bf-113aea2ca67a None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:05.851481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2dfa7-2a1a-4b2e-b7bf-113aea2ca67a None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:05.851996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2dfa7-2a1a-4b2e-b7bf-113aea2ca67a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:05.871922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0cd2dfa7-2a1a-4b2e-b7bf-113aea2ca67a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:05.873135 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0cd2dfa7-2a1a-4b2e-b7bf-113aea2ca67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:05.931865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1331/2650] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:25:05.941780 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:05.943207 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:06.000471 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a53458-404b-4b43-9c79-8c5e445748c0 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:06.004382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321844f1-b9b2-4639-b914-9b1417c7c097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:06.005028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321844f1-b9b2-4639-b914-9b1417c7c097 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.005342 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321844f1-b9b2-4639-b914-9b1417c7c097 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.005614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321844f1-b9b2-4639-b914-9b1417c7c097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.015196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321844f1-b9b2-4639-b914-9b1417c7c097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:06.016667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321844f1-b9b2-4639-b914-9b1417c7c097 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:06.026858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1320/2651] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:06.296872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2dfa7-2a1a-4b2e-b7bf-113aea2ca67a None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:06.298836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0cd2dfa7-2a1a-4b2e-b7bf-113aea2ca67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:06.315347 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2dfa7-2a1a-4b2e-b7bf-113aea2ca67a None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:06.324622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:06.326858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.327424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.327981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] PATH_INFO: `/v3/projects/6915b8ba96034df0811d3a850b5face0` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.329298 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6915b8ba96034df0811d3a850b5face0)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:06.331742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:06.379096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6915b8ba96034df0811d3a850b5face0', 'request_id': 'req-874d856c-f7a4-4a18-aace-bf73df08a923'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:06.385398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6915b8ba96034df0811d3a850b5face0', 'request_id': 'req-874d856c-f7a4-4a18-aace-bf73df08a923'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:06.392144 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6915b8ba96034df0811d3a850b5face0', 'request_id': 'req-874d856c-f7a4-4a18-aace-bf73df08a923'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:06.403199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] The token cache is being invalidate because project 6915b8ba96034df0811d3a850b5face0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:06.403510 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:06.404126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-874d856c-f7a4-4a18-aace-bf73df08a923 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:06.405592 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1332/2652] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:06 2022] DELETE /identity/v3/projects/6915b8ba96034df0811d3a850b5face0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:06.410509 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff388d-f9ab-4f67-baa8-f14f2492958c None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.410788 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff388d-f9ab-4f67-baa8-f14f2492958c None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.411055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff388d-f9ab-4f67-baa8-f14f2492958c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.433013 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7aff388d-f9ab-4f67-baa8-f14f2492958c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:06.434249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7aff388d-f9ab-4f67-baa8-f14f2492958c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:06.452211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:06.495671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1321/2653] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:06.503292 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9e360-c4ec-424b-90f5-f080296f8507 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.503608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9e360-c4ec-424b-90f5-f080296f8507 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.503879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9e360-c4ec-424b-90f5-f080296f8507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.523927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b4c9e360-c4ec-424b-90f5-f080296f8507 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:06.525110 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b4c9e360-c4ec-424b-90f5-f080296f8507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:06.533038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1333/2654] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:06.537197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff388d-f9ab-4f67-baa8-f14f2492958c None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:06.546865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff388d-f9ab-4f67-baa8-f14f2492958c None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:06.551416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febbc15-0bd3-43ad-9f36-ebedd941294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:06.552102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febbc15-0bd3-43ad-9f36-ebedd941294c admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.552376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febbc15-0bd3-43ad-9f36-ebedd941294c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.552644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8febbc15-0bd3-43ad-9f36-ebedd941294c admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.553931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febbc15-0bd3-43ad-9f36-ebedd941294c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:25:06.554231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febbc15-0bd3-43ad-9f36-ebedd941294c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:06.555913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8febbc15-0bd3-43ad-9f36-ebedd941294c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:06.568101 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1322/2655] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:25:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:25:06.572717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9e360-c4ec-424b-90f5-f080296f8507 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:06.582019 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9e360-c4ec-424b-90f5-f080296f8507 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:06.587045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bab40-1729-41d2-af86-5a3af331a47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:06.587701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bab40-1729-41d2-af86-5a3af331a47d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.587981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bab40-1729-41d2-af86-5a3af331a47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.588253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981bab40-1729-41d2-af86-5a3af331a47d admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.588881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bab40-1729-41d2-af86-5a3af331a47d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:06.590793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981bab40-1729-41d2-af86-5a3af331a47d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:06.617982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1334/2656] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:06 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:06.622061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febbc15-0bd3-43ad-9f36-ebedd941294c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:06.632879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8febbc15-0bd3-43ad-9f36-ebedd941294c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:06.689754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d159202-800a-42e2-8223-03c1fdc38924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:06.690359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d159202-800a-42e2-8223-03c1fdc38924 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.690639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d159202-800a-42e2-8223-03c1fdc38924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.690907 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d159202-800a-42e2-8223-03c1fdc38924 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.691586 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d159202-800a-42e2-8223-03c1fdc38924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:06.693709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d159202-800a-42e2-8223-03c1fdc38924 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:06.717007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1323/2657] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:06 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:06.719313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bab40-1729-41d2-af86-5a3af331a47d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:06.730454 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981bab40-1729-41d2-af86-5a3af331a47d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:06.735016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1aadd4-95dc-4696-8432-1f9ff2d1d996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:06.735641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aadd4-95dc-4696-8432-1f9ff2d1d996 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.735912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aadd4-95dc-4696-8432-1f9ff2d1d996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.736178 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1aadd4-95dc-4696-8432-1f9ff2d1d996 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.736804 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1aadd4-95dc-4696-8432-1f9ff2d1d996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:06.738430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1aadd4-95dc-4696-8432-1f9ff2d1d996 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:06.749015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1335/2658] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:06.752937 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d159202-800a-42e2-8223-03c1fdc38924 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:06.763040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d159202-800a-42e2-8223-03c1fdc38924 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:06.767680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466a5b5f-eb82-46f7-b316-10e1140ff060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:06.768264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a5b5f-eb82-46f7-b316-10e1140ff060 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.768540 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a5b5f-eb82-46f7-b316-10e1140ff060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.768809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a5b5f-eb82-46f7-b316-10e1140ff060 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.769442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a5b5f-eb82-46f7-b316-10e1140ff060 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:06.771042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466a5b5f-eb82-46f7-b316-10e1140ff060 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:06.781148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1324/2659] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:06.784968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1aadd4-95dc-4696-8432-1f9ff2d1d996 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:06.795297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1aadd4-95dc-4696-8432-1f9ff2d1d996 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:06.799867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c393cb6-2b58-4dae-bfb6-e3aca8637131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:06.800517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c393cb6-2b58-4dae-bfb6-e3aca8637131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.800788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c393cb6-2b58-4dae-bfb6-e3aca8637131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.801053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c393cb6-2b58-4dae-bfb6-e3aca8637131 admin admin] PATH_INFO: `/v3/projects/0241c7f2dcc841f28cc78d96f8f9a4b8/users/abfbbb6a010846cda2f5edae991558a2/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.815852 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c393cb6-2b58-4dae-bfb6-e3aca8637131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0241c7f2dcc841f28cc78d96f8f9a4b8, user_id=abfbbb6a010846cda2f5edae991558a2, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:06.818614 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c393cb6-2b58-4dae-bfb6-e3aca8637131 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:06.831076 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1336/2660] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:06 2022] PUT /identity/v3/projects/0241c7f2dcc841f28cc78d96f8f9a4b8/users/abfbbb6a010846cda2f5edae991558a2/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:06.843128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57848161-7ecc-48a8-9e51-765bd78a162d None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:06.843437 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57848161-7ecc-48a8-9e51-765bd78a162d None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:06.843756 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57848161-7ecc-48a8-9e51-765bd78a162d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:06.873876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-57848161-7ecc-48a8-9e51-765bd78a162d None None] MFA Rules not processed for user `abfbbb6a010846cda2f5edae991558a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:06.875088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-57848161-7ecc-48a8-9e51-765bd78a162d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:06.945963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1325/2661] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:06.950579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c393cb6-2b58-4dae-bfb6-e3aca8637131 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:06.971189 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:07.013332 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c393cb6-2b58-4dae-bfb6-e3aca8637131 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:07.017950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33af727-38d3-4409-ad00-497eb8a2137e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:07.018547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33af727-38d3-4409-ad00-497eb8a2137e admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:07.018828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33af727-38d3-4409-ad00-497eb8a2137e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:07.019101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33af727-38d3-4409-ad00-497eb8a2137e admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:07.019769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33af727-38d3-4409-ad00-497eb8a2137e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:07.022354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33af727-38d3-4409-ad00-497eb8a2137e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:07.048300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1337/2662] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:06 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:07.052394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57848161-7ecc-48a8-9e51-765bd78a162d None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:07.063180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57848161-7ecc-48a8-9e51-765bd78a162d None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:07.075157 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:07.120183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b784594-1440-41e3-b79f-9061b48c3cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:07.120784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b784594-1440-41e3-b79f-9061b48c3cfd admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:07.121061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b784594-1440-41e3-b79f-9061b48c3cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:07.121331 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b784594-1440-41e3-b79f-9061b48c3cfd admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:07.121995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b784594-1440-41e3-b79f-9061b48c3cfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:07.123966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b784594-1440-41e3-b79f-9061b48c3cfd admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:07.144825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1326/2663] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:07 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:25:07.149008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33af727-38d3-4409-ad00-497eb8a2137e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:07.159571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33af727-38d3-4409-ad00-497eb8a2137e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:07.164311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39521586-45aa-479a-8303-fb2344ccd8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:07.164932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39521586-45aa-479a-8303-fb2344ccd8fd admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:07.165210 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39521586-45aa-479a-8303-fb2344ccd8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:07.165480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39521586-45aa-479a-8303-fb2344ccd8fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:07.166121 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39521586-45aa-479a-8303-fb2344ccd8fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:07.167766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39521586-45aa-479a-8303-fb2344ccd8fd admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:07.178796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1338/2664] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:07.183060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b784594-1440-41e3-b79f-9061b48c3cfd admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:07.192847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b784594-1440-41e3-b79f-9061b48c3cfd admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:07.198226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaf36d-3d8c-4775-ab49-5e9abcbec0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:07.198865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaf36d-3d8c-4775-ab49-5e9abcbec0a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:07.199137 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaf36d-3d8c-4775-ab49-5e9abcbec0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:07.199405 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaaf36d-3d8c-4775-ab49-5e9abcbec0a0 admin admin] PATH_INFO: `/v3/projects/9350f3f31af041559d892ccbd0ae86e8/users/3a5faad7a44044ac91079caca0557ee3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:07.214338 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaaf36d-3d8c-4775-ab49-5e9abcbec0a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9350f3f31af041559d892ccbd0ae86e8, user_id=3a5faad7a44044ac91079caca0557ee3, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:07.217040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaaf36d-3d8c-4775-ab49-5e9abcbec0a0 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:07.230309 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1327/2665] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:07 2022] PUT /identity/v3/projects/9350f3f31af041559d892ccbd0ae86e8/users/3a5faad7a44044ac91079caca0557ee3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:07.234444 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39521586-45aa-479a-8303-fb2344ccd8fd admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:07.245340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39521586-45aa-479a-8303-fb2344ccd8fd admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:07.300293 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3d48a2-70c8-4994-ae5b-0a33dd0e16d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:07.300944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d48a2-70c8-4994-ae5b-0a33dd0e16d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:07.301158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d48a2-70c8-4994-ae5b-0a33dd0e16d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:07.301502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3d48a2-70c8-4994-ae5b-0a33dd0e16d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:07.302140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d48a2-70c8-4994-ae5b-0a33dd0e16d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:07.304317 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3d48a2-70c8-4994-ae5b-0a33dd0e16d1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:07.313881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1339/2666] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:07 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:07.318021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaf36d-3d8c-4775-ab49-5e9abcbec0a0 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:07.329058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaaf36d-3d8c-4775-ab49-5e9abcbec0a0 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:07.333881 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a8874-86e3-4d15-a793-cc611c80750e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:07.334618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a8874-86e3-4d15-a793-cc611c80750e admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:07.334890 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a8874-86e3-4d15-a793-cc611c80750e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:07.335161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5a8874-86e3-4d15-a793-cc611c80750e admin admin] PATH_INFO: `/v3/projects/9350f3f31af041559d892ccbd0ae86e8/users/3a5faad7a44044ac91079caca0557ee3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:07.337200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a8874-86e3-4d15-a793-cc611c80750e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9350f3f31af041559d892ccbd0ae86e8, user_id=3a5faad7a44044ac91079caca0557ee3, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:07.339936 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5a8874-86e3-4d15-a793-cc611c80750e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:07.351623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1328/2667] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:07 2022] PUT /identity/v3/projects/9350f3f31af041559d892ccbd0ae86e8/users/3a5faad7a44044ac91079caca0557ee3/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:07.362438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4cadc-44a7-4372-be58-49d3537a1d78 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:07.362828 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4cadc-44a7-4372-be58-49d3537a1d78 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:07.363115 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac4cadc-44a7-4372-be58-49d3537a1d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:07.394157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3ac4cadc-44a7-4372-be58-49d3537a1d78 None None] MFA Rules not processed for user `3a5faad7a44044ac91079caca0557ee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:07.395366 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3ac4cadc-44a7-4372-be58-49d3537a1d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:07.472661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1340/2668] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:07 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:07.485383 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a8874-86e3-4d15-a793-cc611c80750e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:07.487045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5c5a8874-86e3-4d15-a793-cc611c80750e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:07.551233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5a8874-86e3-4d15-a793-cc611c80750e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:07.556791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51721cc-db01-45b5-949f-05ed58c9f956 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:07.557446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51721cc-db01-45b5-949f-05ed58c9f956 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:07.557731 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51721cc-db01-45b5-949f-05ed58c9f956 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:07.558064 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51721cc-db01-45b5-949f-05ed58c9f956 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:07.569370 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51721cc-db01-45b5-949f-05ed58c9f956 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:07.570885 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51721cc-db01-45b5-949f-05ed58c9f956 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:07.580793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1329/2669] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:07 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:08.994622 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4cadc-44a7-4372-be58-49d3537a1d78 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:09.011598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:09.057222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac4cadc-44a7-4372-be58-49d3537a1d78 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:09.061707 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:09.062325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:09.062599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:09.062889 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] PATH_INFO: `/v3/users/abfbbb6a010846cda2f5edae991558a2` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:09.064002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] RBAC: Authorizing `identity:delete_user(user_id=abfbbb6a010846cda2f5edae991558a2)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:09.065941 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:09.112926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abfbbb6a010846cda2f5edae991558a2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.121182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abfbbb6a010846cda2f5edae991558a2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.121799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abfbbb6a010846cda2f5edae991558a2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.122449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abfbbb6a010846cda2f5edae991558a2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.129337 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abfbbb6a010846cda2f5edae991558a2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.146348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1341/2670] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:08 2022] DELETE /identity/v3/users/abfbbb6a010846cda2f5edae991558a2 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:09.151311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abdb4a-5296-422f-b79d-ad7b4919b264 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:09.151815 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abdb4a-5296-422f-b79d-ad7b4919b264 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:09.151923 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abdb4a-5296-422f-b79d-ad7b4919b264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:09.170967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-b6abdb4a-5296-422f-b79d-ad7b4919b264 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:09.172186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b6abdb4a-5296-422f-b79d-ad7b4919b264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:09.229454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1330/2671] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:09.239531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:09.241640 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:09.298453 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47b2a8-2cf2-4621-893f-a81dc5df4fff admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:09.302134 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6c972-f7ab-4de6-a0e6-103679b45360 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:09.302751 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6c972-f7ab-4de6-a0e6-103679b45360 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:09.303023 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6c972-f7ab-4de6-a0e6-103679b45360 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:09.303501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f6c972-f7ab-4de6-a0e6-103679b45360 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:09.312533 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6c972-f7ab-4de6-a0e6-103679b45360 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:09.313959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f6c972-f7ab-4de6-a0e6-103679b45360 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:09.323400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1342/2672] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:09.596178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abdb4a-5296-422f-b79d-ad7b4919b264 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:09.597648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-b6abdb4a-5296-422f-b79d-ad7b4919b264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:09.611876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abdb4a-5296-422f-b79d-ad7b4919b264 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:09.617350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:09.618474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:09.618753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:09.619187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] PATH_INFO: `/v3/projects/0241c7f2dcc841f28cc78d96f8f9a4b8` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:09.620674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=0241c7f2dcc841f28cc78d96f8f9a4b8)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:09.623136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:09.632007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:09.670737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0241c7f2dcc841f28cc78d96f8f9a4b8', 'request_id': 'req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.677264 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0241c7f2dcc841f28cc78d96f8f9a4b8', 'request_id': 'req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.683703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0241c7f2dcc841f28cc78d96f8f9a4b8', 'request_id': 'req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.694867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] The token cache is being invalidate because project 0241c7f2dcc841f28cc78d96f8f9a4b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:09.695151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.695807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.697282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1331/2673] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:09 2022] DELETE /identity/v3/projects/0241c7f2dcc841f28cc78d96f8f9a4b8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:09.701339 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6c972-f7ab-4de6-a0e6-103679b45360 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:09.711108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f6c972-f7ab-4de6-a0e6-103679b45360 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:09.744583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:09.769620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:09.771932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:09.771932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:09.771932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] PATH_INFO: `/v3/users/3a5faad7a44044ac91079caca0557ee3` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:09.772452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a5faad7a44044ac91079caca0557ee3)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:09.774379 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:09.821120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a5faad7a44044ac91079caca0557ee3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.828245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a5faad7a44044ac91079caca0557ee3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.828859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a5faad7a44044ac91079caca0557ee3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.829457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a5faad7a44044ac91079caca0557ee3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.836893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8391aa11-95e9-4a56-b458-6f27a7c3309e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a5faad7a44044ac91079caca0557ee3'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:09.854638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1343/2674] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:09 2022] DELETE /identity/v3/users/3a5faad7a44044ac91079caca0557ee3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:10.122866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:10.124320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:10.137417 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97fdcc1-6358-451c-b19b-d5b60c8cd9ae admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:10.194444 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:10.195126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.195670 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.195904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] PATH_INFO: `/v3/projects/9350f3f31af041559d892ccbd0ae86e8` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.197029 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9350f3f31af041559d892ccbd0ae86e8)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:10.198922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:10.245640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9350f3f31af041559d892ccbd0ae86e8', 'request_id': 'req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:10.252880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9350f3f31af041559d892ccbd0ae86e8', 'request_id': 'req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:10.258784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9350f3f31af041559d892ccbd0ae86e8', 'request_id': 'req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:10.270610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] The token cache is being invalidate because project 9350f3f31af041559d892ccbd0ae86e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:10.270927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:10.271588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-bd95ad13-3ccd-4b64-87c6-26f995c1e70e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:10.273591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1332/2675] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:10 2022] DELETE /identity/v3/projects/9350f3f31af041559d892ccbd0ae86e8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:10.278587 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9bd9-9e93-4c36-8a21-2bea0540d310 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.279058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9bd9-9e93-4c36-8a21-2bea0540d310 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.279173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca9bd9-9e93-4c36-8a21-2bea0540d310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.299460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fdca9bd9-9e93-4c36-8a21-2bea0540d310 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:10.300679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fdca9bd9-9e93-4c36-8a21-2bea0540d310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:10.361039 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1344/2676] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:10.369871 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fd37f-e427-402d-863c-7837e85346bc None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.370089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fd37f-e427-402d-863c-7837e85346bc None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.370358 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1fd37f-e427-402d-863c-7837e85346bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.389534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-3e1fd37f-e427-402d-863c-7837e85346bc None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:10.390721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-3e1fd37f-e427-402d-863c-7837e85346bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:10.398588 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1333/2677] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:10.402700 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9bd9-9e93-4c36-8a21-2bea0540d310 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:10.412505 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca9bd9-9e93-4c36-8a21-2bea0540d310 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:10.417848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb6e9e-e3bb-4435-98a5-6affa8fb9965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:10.418447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6e9e-e3bb-4435-98a5-6affa8fb9965 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.418726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6e9e-e3bb-4435-98a5-6affa8fb9965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.418995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb6e9e-e3bb-4435-98a5-6affa8fb9965 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.419686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb6e9e-e3bb-4435-98a5-6affa8fb9965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:25:10.419987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb6e9e-e3bb-4435-98a5-6affa8fb9965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:10.421582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fb6e9e-e3bb-4435-98a5-6affa8fb9965 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:10.434247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1345/2678] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:25:10.438430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fd37f-e427-402d-863c-7837e85346bc None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:10.447737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1fd37f-e427-402d-863c-7837e85346bc None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:10.452446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f5902-7e4b-49c9-b4d4-ff3212afd7b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:10.453043 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f5902-7e4b-49c9-b4d4-ff3212afd7b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.453314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f5902-7e4b-49c9-b4d4-ff3212afd7b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.453581 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f5902-7e4b-49c9-b4d4-ff3212afd7b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.454212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f5902-7e4b-49c9-b4d4-ff3212afd7b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:10.456196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71f5902-7e4b-49c9-b4d4-ff3212afd7b9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:10.478496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:10.487357 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1334/2679] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:10 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:10.492539 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb6e9e-e3bb-4435-98a5-6affa8fb9965 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:10.502536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb6e9e-e3bb-4435-98a5-6affa8fb9965 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:10.543715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:10.560045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03270614-7c85-4919-b007-43045b1aff26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:10.560638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03270614-7c85-4919-b007-43045b1aff26 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.560914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03270614-7c85-4919-b007-43045b1aff26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.561185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03270614-7c85-4919-b007-43045b1aff26 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.561811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03270614-7c85-4919-b007-43045b1aff26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:10.563759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03270614-7c85-4919-b007-43045b1aff26 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:10.585771 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1346/2680] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:10 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:10.589934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f5902-7e4b-49c9-b4d4-ff3212afd7b9 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:10.600377 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f5902-7e4b-49c9-b4d4-ff3212afd7b9 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:10.605600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fc78e-9e34-4701-98ca-48b96d4c19c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:10.606194 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fc78e-9e34-4701-98ca-48b96d4c19c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.606496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fc78e-9e34-4701-98ca-48b96d4c19c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.606762 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fc78e-9e34-4701-98ca-48b96d4c19c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.607393 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fc78e-9e34-4701-98ca-48b96d4c19c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:10.609032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69fc78e-9e34-4701-98ca-48b96d4c19c3 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:10.620322 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1335/2681] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:10.624098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03270614-7c85-4919-b007-43045b1aff26 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:10.634590 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03270614-7c85-4919-b007-43045b1aff26 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:10.639381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6f9295-13b1-443a-86fb-3ec8899160e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:10.640153 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f9295-13b1-443a-86fb-3ec8899160e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.640469 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f9295-13b1-443a-86fb-3ec8899160e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.640780 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6f9295-13b1-443a-86fb-3ec8899160e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.641534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f9295-13b1-443a-86fb-3ec8899160e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:10.643140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6f9295-13b1-443a-86fb-3ec8899160e6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:10.654380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1347/2682] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:10.658438 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fc78e-9e34-4701-98ca-48b96d4c19c3 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:10.668836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69fc78e-9e34-4701-98ca-48b96d4c19c3 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:10.673456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe7735-4e8e-414c-a06a-83bef0345c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:10.674100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe7735-4e8e-414c-a06a-83bef0345c44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.674378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe7735-4e8e-414c-a06a-83bef0345c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.674650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe7735-4e8e-414c-a06a-83bef0345c44 admin admin] PATH_INFO: `/v3/projects/97b28a0e840b4aebbd975e1eb1753765/users/0263bf4da7124ac589518d0fd8a913af/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.689212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe7735-4e8e-414c-a06a-83bef0345c44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97b28a0e840b4aebbd975e1eb1753765, user_id=0263bf4da7124ac589518d0fd8a913af, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:10.691927 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbe7735-4e8e-414c-a06a-83bef0345c44 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:10.705253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1336/2683] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:10 2022] PUT /identity/v3/projects/97b28a0e840b4aebbd975e1eb1753765/users/0263bf4da7124ac589518d0fd8a913af/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:10.717464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7b3f6-ad80-4e1f-8e2c-50a99f6df97f None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.717843 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7b3f6-ad80-4e1f-8e2c-50a99f6df97f None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.718117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7b3f6-ad80-4e1f-8e2c-50a99f6df97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.748527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-cbb7b3f6-ad80-4e1f-8e2c-50a99f6df97f None None] MFA Rules not processed for user `0263bf4da7124ac589518d0fd8a913af`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:10.749724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cbb7b3f6-ad80-4e1f-8e2c-50a99f6df97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:10.815824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1348/2684] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:10 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:10.820919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe7735-4e8e-414c-a06a-83bef0345c44 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:10.883878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe7735-4e8e-414c-a06a-83bef0345c44 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:10.888456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de84c24-d10c-4c09-b9a5-007f9599fd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:10.889057 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84c24-d10c-4c09-b9a5-007f9599fd02 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.889336 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84c24-d10c-4c09-b9a5-007f9599fd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.889610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de84c24-d10c-4c09-b9a5-007f9599fd02 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.890244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de84c24-d10c-4c09-b9a5-007f9599fd02 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:10.892203 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de84c24-d10c-4c09-b9a5-007f9599fd02 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:10.918060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1337/2685] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:10 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:10.922462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7b3f6-ad80-4e1f-8e2c-50a99f6df97f None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:10.933630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7b3f6-ad80-4e1f-8e2c-50a99f6df97f None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:10.990688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884b36f-47e6-4ee0-9fd8-9516452e3022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:10.991282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884b36f-47e6-4ee0-9fd8-9516452e3022 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:10.991591 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884b36f-47e6-4ee0-9fd8-9516452e3022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:10.992125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884b36f-47e6-4ee0-9fd8-9516452e3022 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:10.992865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884b36f-47e6-4ee0-9fd8-9516452e3022 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:10.994776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6884b36f-47e6-4ee0-9fd8-9516452e3022 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:11.016114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1349/2686] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:10 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:25:11.020104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de84c24-d10c-4c09-b9a5-007f9599fd02 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:11.030905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de84c24-d10c-4c09-b9a5-007f9599fd02 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:11.035677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6f2f0-0c8e-4f2d-99f8-06c5c5080c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:11.036267 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6f2f0-0c8e-4f2d-99f8-06c5c5080c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:11.036541 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6f2f0-0c8e-4f2d-99f8-06c5c5080c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:11.036810 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f6f2f0-0c8e-4f2d-99f8-06c5c5080c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:11.037449 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6f2f0-0c8e-4f2d-99f8-06c5c5080c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:11.040136 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f6f2f0-0c8e-4f2d-99f8-06c5c5080c78 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:11.050499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1338/2687] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:11.055406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884b36f-47e6-4ee0-9fd8-9516452e3022 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:11.065253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884b36f-47e6-4ee0-9fd8-9516452e3022 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:11.069850 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697ab38-9f59-4d85-ab5a-a7be67d8d0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:11.071104 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697ab38-9f59-4d85-ab5a-a7be67d8d0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:11.071384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697ab38-9f59-4d85-ab5a-a7be67d8d0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:11.071692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8697ab38-9f59-4d85-ab5a-a7be67d8d0a9 admin admin] PATH_INFO: `/v3/projects/8785953894724fc798cf4788872a8beb/users/05d9bcd2bd174f429b21d70586612e6d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:11.086272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697ab38-9f59-4d85-ab5a-a7be67d8d0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8785953894724fc798cf4788872a8beb, user_id=05d9bcd2bd174f429b21d70586612e6d, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:11.089050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8697ab38-9f59-4d85-ab5a-a7be67d8d0a9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:11.102369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1350/2688] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:11 2022] PUT /identity/v3/projects/8785953894724fc798cf4788872a8beb/users/05d9bcd2bd174f429b21d70586612e6d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:11.106261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6f2f0-0c8e-4f2d-99f8-06c5c5080c78 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:11.116611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f6f2f0-0c8e-4f2d-99f8-06c5c5080c78 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:11.172287 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2585c19-9fd9-4fd5-a221-eeff50d22381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:11.172901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2585c19-9fd9-4fd5-a221-eeff50d22381 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:11.173179 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2585c19-9fd9-4fd5-a221-eeff50d22381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:11.173454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2585c19-9fd9-4fd5-a221-eeff50d22381 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:11.174087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2585c19-9fd9-4fd5-a221-eeff50d22381 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:11.175719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2585c19-9fd9-4fd5-a221-eeff50d22381 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:11.186335 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1339/2689] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:11.190322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697ab38-9f59-4d85-ab5a-a7be67d8d0a9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:11.200399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8697ab38-9f59-4d85-ab5a-a7be67d8d0a9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:11.204991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20999a6b-e136-44e2-8db3-e933f661bf1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:11.205632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20999a6b-e136-44e2-8db3-e933f661bf1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:11.205910 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20999a6b-e136-44e2-8db3-e933f661bf1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:11.206183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20999a6b-e136-44e2-8db3-e933f661bf1e admin admin] PATH_INFO: `/v3/projects/8785953894724fc798cf4788872a8beb/users/05d9bcd2bd174f429b21d70586612e6d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:11.208143 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20999a6b-e136-44e2-8db3-e933f661bf1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8785953894724fc798cf4788872a8beb, user_id=05d9bcd2bd174f429b21d70586612e6d, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:11.210809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20999a6b-e136-44e2-8db3-e933f661bf1e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:11.220761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1351/2690] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:11 2022] PUT /identity/v3/projects/8785953894724fc798cf4788872a8beb/users/05d9bcd2bd174f429b21d70586612e6d/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:11.231478 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c410f34-9a9a-4a64-a9bf-5c257edaa1ff None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:11.231974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c410f34-9a9a-4a64-a9bf-5c257edaa1ff None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:11.232083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c410f34-9a9a-4a64-a9bf-5c257edaa1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:11.260428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-9c410f34-9a9a-4a64-a9bf-5c257edaa1ff None None] MFA Rules not processed for user `05d9bcd2bd174f429b21d70586612e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:11.261662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9c410f34-9a9a-4a64-a9bf-5c257edaa1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:11.281201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:11.330184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1340/2691] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:11 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:11.342984 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20999a6b-e136-44e2-8db3-e933f661bf1e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:11.344745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-20999a6b-e136-44e2-8db3-e933f661bf1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:11.351035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:11.404529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20999a6b-e136-44e2-8db3-e933f661bf1e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:11.408159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640ea1f-de38-48ff-866e-35b26ddc1757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:11.408748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640ea1f-de38-48ff-866e-35b26ddc1757 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:11.409017 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640ea1f-de38-48ff-866e-35b26ddc1757 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:11.409284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640ea1f-de38-48ff-866e-35b26ddc1757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:11.418560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640ea1f-de38-48ff-866e-35b26ddc1757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:11.420015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3640ea1f-de38-48ff-866e-35b26ddc1757 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:11.430199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1352/2692] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:11 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:11.748046 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c410f34-9a9a-4a64-a9bf-5c257edaa1ff None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:11.757068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c410f34-9a9a-4a64-a9bf-5c257edaa1ff None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:11.761751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b92e71-38cd-4cf8-a6d6-ef809063a327 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d9bcd2bd174f429b21d70586612e6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8785953894724fc798cf4788872a8beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:11.762351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92e71-38cd-4cf8-a6d6-ef809063a327 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:11.762637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92e71-38cd-4cf8-a6d6-ef809063a327 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:11.762911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b92e71-38cd-4cf8-a6d6-ef809063a327 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:11.763598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b92e71-38cd-4cf8-a6d6-ef809063a327 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:25:11.763904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b92e71-38cd-4cf8-a6d6-ef809063a327 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:11.765503 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b92e71-38cd-4cf8-a6d6-ef809063a327 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:11.777312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1341/2693] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:25:11.781435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640ea1f-de38-48ff-866e-35b26ddc1757 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:11.790589 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640ea1f-de38-48ff-866e-35b26ddc1757 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:11.795158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a83ca-998c-4fc0-b2a0-699d33d8f56e tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d9bcd2bd174f429b21d70586612e6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8785953894724fc798cf4788872a8beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:11.795776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a83ca-998c-4fc0-b2a0-699d33d8f56e tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:11.796053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a83ca-998c-4fc0-b2a0-699d33d8f56e tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:11.796324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01a83ca-998c-4fc0-b2a0-699d33d8f56e tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:11.796954 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a83ca-998c-4fc0-b2a0-699d33d8f56e tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:11.798855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01a83ca-998c-4fc0-b2a0-699d33d8f56e tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:11.826015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1353/2694] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:11 2022] POST /identity/v3/projects => generated 354 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:11.830217 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b92e71-38cd-4cf8-a6d6-ef809063a327 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:11.841125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b92e71-38cd-4cf8-a6d6-ef809063a327 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:11.899431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e79bc7-b8d3-4d1d-b7eb-70e8f49d3674 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d9bcd2bd174f429b21d70586612e6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8785953894724fc798cf4788872a8beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:11.900052 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e79bc7-b8d3-4d1d-b7eb-70e8f49d3674 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:11.900333 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e79bc7-b8d3-4d1d-b7eb-70e8f49d3674 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:11.900609 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e79bc7-b8d3-4d1d-b7eb-70e8f49d3674 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:11.901249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e79bc7-b8d3-4d1d-b7eb-70e8f49d3674 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:25:11.901553 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e79bc7-b8d3-4d1d-b7eb-70e8f49d3674 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:11.903185 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e79bc7-b8d3-4d1d-b7eb-70e8f49d3674 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:11.914951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1342/2695] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:25:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:25:11.956749 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a83ca-998c-4fc0-b2a0-699d33d8f56e tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:11.967794 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01a83ca-998c-4fc0-b2a0-699d33d8f56e tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:11.972265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '05d9bcd2bd174f429b21d70586612e6d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8785953894724fc798cf4788872a8beb', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:11.972873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:11.973147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:11.973411 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] PATH_INFO: `/v3/projects/6dc560a7cdb8461186b8cb3a41c1ce84` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:11.974546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6dc560a7cdb8461186b8cb3a41c1ce84)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:11.976817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:12.022675 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dc560a7cdb8461186b8cb3a41c1ce84', 'request_id': 'req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.028378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dc560a7cdb8461186b8cb3a41c1ce84', 'request_id': 'req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.035541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dc560a7cdb8461186b8cb3a41c1ce84', 'request_id': 'req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.047722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] The token cache is being invalidate because project 6dc560a7cdb8461186b8cb3a41c1ce84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:12.048010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.048659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-f0ecfc16-fb24-4b1a-a69a-0a62f7b7b750 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.050141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1354/2696] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:11 2022] DELETE /identity/v3/projects/6dc560a7cdb8461186b8cb3a41c1ce84 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:12.237244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e79bc7-b8d3-4d1d-b7eb-70e8f49d3674 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:12.239258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f2e79bc7-b8d3-4d1d-b7eb-70e8f49d3674 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:12.303111 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e79bc7-b8d3-4d1d-b7eb-70e8f49d3674 tempest-VolumeQuotaClassesTest-1870825995 tempest-VolumeQuotaClassesTest-1870825995-project-admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:12.344867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:12.357431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:12.358050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:12.358329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:12.358600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] PATH_INFO: `/v3/users/0263bf4da7124ac589518d0fd8a913af` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:12.359751 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0263bf4da7124ac589518d0fd8a913af)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:12.361660 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:12.407125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0263bf4da7124ac589518d0fd8a913af'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.407768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0263bf4da7124ac589518d0fd8a913af'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.415080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0263bf4da7124ac589518d0fd8a913af'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.421869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0263bf4da7124ac589518d0fd8a913af'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.422544 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0263bf4da7124ac589518d0fd8a913af'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.447008 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1343/2697] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:12 2022] DELETE /identity/v3/users/0263bf4da7124ac589518d0fd8a913af => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:12.451759 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bbf1d-5d69-4b67-9aab-143e5f27d04c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:12.452225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bbf1d-5d69-4b67-9aab-143e5f27d04c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:12.452330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429bbf1d-5d69-4b67-9aab-143e5f27d04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:12.470952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-429bbf1d-5d69-4b67-9aab-143e5f27d04c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:12.472100 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-429bbf1d-5d69-4b67-9aab-143e5f27d04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:12.529611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1355/2698] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:12.539394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:12.540812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:12.600031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41737505-9e3e-4cc3-aa91-15b011581735 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:12.604440 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f662e8-14b0-4363-9213-3249c96d1943 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:12.605022 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f662e8-14b0-4363-9213-3249c96d1943 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:12.605300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f662e8-14b0-4363-9213-3249c96d1943 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:12.605570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f662e8-14b0-4363-9213-3249c96d1943 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:12.614915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f662e8-14b0-4363-9213-3249c96d1943 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:12.616379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f662e8-14b0-4363-9213-3249c96d1943 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:12.626374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1344/2699] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:12.899085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bbf1d-5d69-4b67-9aab-143e5f27d04c None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:12.900579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-429bbf1d-5d69-4b67-9aab-143e5f27d04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:12.912671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429bbf1d-5d69-4b67-9aab-143e5f27d04c None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:12.917809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:12.918443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:12.918722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:12.918995 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] PATH_INFO: `/v3/projects/97b28a0e840b4aebbd975e1eb1753765` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:12.921524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=97b28a0e840b4aebbd975e1eb1753765)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:12.923565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:12.926278 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:12.973803 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97b28a0e840b4aebbd975e1eb1753765', 'request_id': 'req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.979458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97b28a0e840b4aebbd975e1eb1753765', 'request_id': 'req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.986545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97b28a0e840b4aebbd975e1eb1753765', 'request_id': 'req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.998486 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] The token cache is being invalidate because project 97b28a0e840b4aebbd975e1eb1753765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:12.998773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:12.999427 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.000966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1356/2700] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:12 2022] DELETE /identity/v3/projects/97b28a0e840b4aebbd975e1eb1753765 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:13.004934 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f662e8-14b0-4363-9213-3249c96d1943 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:13.015430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f662e8-14b0-4363-9213-3249c96d1943 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:13.070951 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:13.071599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:13.071874 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:13.072143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] PATH_INFO: `/v3/users/05d9bcd2bd174f429b21d70586612e6d` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:13.073243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05d9bcd2bd174f429b21d70586612e6d)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:13.075143 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:13.085372 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:13.122173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05d9bcd2bd174f429b21d70586612e6d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.122806 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05d9bcd2bd174f429b21d70586612e6d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.129671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05d9bcd2bd174f429b21d70586612e6d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.137007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05d9bcd2bd174f429b21d70586612e6d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.137681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-79b1f042-dc01-47b6-ba9f-fb8804e2a890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05d9bcd2bd174f429b21d70586612e6d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.155132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1345/2701] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:13 2022] DELETE /identity/v3/users/05d9bcd2bd174f429b21d70586612e6d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:13.432944 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:13.439363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:13.453609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d2c45b-af39-4e14-bce2-f92c5828b6cc admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:13.511410 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:13.512008 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:13.512285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:13.512556 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] PATH_INFO: `/v3/projects/8785953894724fc798cf4788872a8beb` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:13.513662 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8785953894724fc798cf4788872a8beb)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:13.515620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:13.561196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8785953894724fc798cf4788872a8beb', 'request_id': 'req-10e12608-f739-4252-b8f9-612604d11873'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.566736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8785953894724fc798cf4788872a8beb', 'request_id': 'req-10e12608-f739-4252-b8f9-612604d11873'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.573788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8785953894724fc798cf4788872a8beb', 'request_id': 'req-10e12608-f739-4252-b8f9-612604d11873'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.584572 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] The token cache is being invalidate because project 8785953894724fc798cf4788872a8beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:13.584883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.585558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-10e12608-f739-4252-b8f9-612604d11873 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:13.587035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1357/2702] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:13 2022] DELETE /identity/v3/projects/8785953894724fc798cf4788872a8beb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:13.592974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ba93c-9f16-4e62-892a-59e9f6dd1294 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:13.593430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ba93c-9f16-4e62-892a-59e9f6dd1294 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:13.593720 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ba93c-9f16-4e62-892a-59e9f6dd1294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:13.613963 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-464ba93c-9f16-4e62-892a-59e9f6dd1294 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:13.615163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-464ba93c-9f16-4e62-892a-59e9f6dd1294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:13.687036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1346/2703] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:13.695827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19b8b3-f998-4970-b923-a32e2a7c4913 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:13.696108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19b8b3-f998-4970-b923-a32e2a7c4913 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:13.696382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19b8b3-f998-4970-b923-a32e2a7c4913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:13.716461 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4f19b8b3-f998-4970-b923-a32e2a7c4913 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:13.717663 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4f19b8b3-f998-4970-b923-a32e2a7c4913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:13.726670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1358/2704] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:13.731497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ba93c-9f16-4e62-892a-59e9f6dd1294 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:13.741271 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ba93c-9f16-4e62-892a-59e9f6dd1294 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:13.746224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c4e3f-822a-4340-a46d-0390d9b77248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:13.746809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c4e3f-822a-4340-a46d-0390d9b77248 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:13.747082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c4e3f-822a-4340-a46d-0390d9b77248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:13.747348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c4e3f-822a-4340-a46d-0390d9b77248 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:13.748030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c4e3f-822a-4340-a46d-0390d9b77248 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:25:13.748327 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c4e3f-822a-4340-a46d-0390d9b77248 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:13.749914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c4e3f-822a-4340-a46d-0390d9b77248 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:13.761598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1347/2705] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:25:13.765727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19b8b3-f998-4970-b923-a32e2a7c4913 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:13.775557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19b8b3-f998-4970-b923-a32e2a7c4913 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:13.780186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34c5ac-94f0-4159-badc-952f22f3c610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:13.780767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34c5ac-94f0-4159-badc-952f22f3c610 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:13.781042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34c5ac-94f0-4159-badc-952f22f3c610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:13.781310 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef34c5ac-94f0-4159-badc-952f22f3c610 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:13.781935 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34c5ac-94f0-4159-badc-952f22f3c610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:13.783898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef34c5ac-94f0-4159-badc-952f22f3c610 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:13.799083 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:13.812711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1359/2706] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:13 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:13.817224 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c4e3f-822a-4340-a46d-0390d9b77248 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:13.827155 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c4e3f-822a-4340-a46d-0390d9b77248 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:13.884517 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a061a7-f35f-4333-865a-ac3eedfbeb48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:13.885128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a061a7-f35f-4333-865a-ac3eedfbeb48 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:13.885403 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a061a7-f35f-4333-865a-ac3eedfbeb48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:13.885671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a061a7-f35f-4333-865a-ac3eedfbeb48 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:13.886296 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a061a7-f35f-4333-865a-ac3eedfbeb48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:13.888249 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a061a7-f35f-4333-865a-ac3eedfbeb48 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:13.916082 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1348/2707] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:13 2022] POST /identity/v3/users => generated 304 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:25:13.920297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34c5ac-94f0-4159-badc-952f22f3c610 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:13.931496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef34c5ac-94f0-4159-badc-952f22f3c610 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:13.936141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd4ce0-b074-4997-89d7-c69d2a801240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:13.936735 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd4ce0-b074-4997-89d7-c69d2a801240 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:13.937013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd4ce0-b074-4997-89d7-c69d2a801240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:13.937285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bd4ce0-b074-4997-89d7-c69d2a801240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:13.937927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd4ce0-b074-4997-89d7-c69d2a801240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:13.939577 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bd4ce0-b074-4997-89d7-c69d2a801240 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:13.949770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1360/2708] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:13.953914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a061a7-f35f-4333-865a-ac3eedfbeb48 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:13.960182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:13.965009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a061a7-f35f-4333-865a-ac3eedfbeb48 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:13.971394 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cd3032-72a3-47b3-8ada-38a50f34a4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:13.972016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd3032-72a3-47b3-8ada-38a50f34a4d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:13.972295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd3032-72a3-47b3-8ada-38a50f34a4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:13.973167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cd3032-72a3-47b3-8ada-38a50f34a4d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:13.974012 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd3032-72a3-47b3-8ada-38a50f34a4d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:13.976725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cd3032-72a3-47b3-8ada-38a50f34a4d9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:13.987563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1349/2709] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:13.992117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd4ce0-b074-4997-89d7-c69d2a801240 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:14.002111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bd4ce0-b074-4997-89d7-c69d2a801240 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:14.006711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c52d-0c4c-4631-9230-023d34f6e118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:14.007354 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c52d-0c4c-4631-9230-023d34f6e118 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.007671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c52d-0c4c-4631-9230-023d34f6e118 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.007942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f9c52d-0c4c-4631-9230-023d34f6e118 admin admin] PATH_INFO: `/v3/projects/f6fa2f8b1595409285ca525d27be774d/users/68216817a6884853a1a5449d7e675887/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.022137 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9c52d-0c4c-4631-9230-023d34f6e118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6fa2f8b1595409285ca525d27be774d, user_id=68216817a6884853a1a5449d7e675887, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:14.025517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f9c52d-0c4c-4631-9230-023d34f6e118 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:14.038001 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1361/2710] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:13 2022] PUT /identity/v3/projects/f6fa2f8b1595409285ca525d27be774d/users/68216817a6884853a1a5449d7e675887/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:14.049165 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d20d74-d468-4605-95eb-8dc53c70e896 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.049468 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d20d74-d468-4605-95eb-8dc53c70e896 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.049739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d20d74-d468-4605-95eb-8dc53c70e896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.076959 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-29d20d74-d468-4605-95eb-8dc53c70e896 None None] MFA Rules not processed for user `68216817a6884853a1a5449d7e675887`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:14.078134 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-29d20d74-d468-4605-95eb-8dc53c70e896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:14.141603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1350/2711] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:14 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:14.146213 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c52d-0c4c-4631-9230-023d34f6e118 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:14.207223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f9c52d-0c4c-4631-9230-023d34f6e118 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:14.212498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8b50e-121b-49e1-b612-3673ddfa1d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:14.213085 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8b50e-121b-49e1-b612-3673ddfa1d4e admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.213358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8b50e-121b-49e1-b612-3673ddfa1d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.213625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd8b50e-121b-49e1-b612-3673ddfa1d4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.214245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8b50e-121b-49e1-b612-3673ddfa1d4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:14.216182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd8b50e-121b-49e1-b612-3673ddfa1d4e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:14.243151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1362/2712] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:14 2022] POST /identity/v3/projects => generated 407 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:14.247242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d20d74-d468-4605-95eb-8dc53c70e896 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:14.258512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d20d74-d468-4605-95eb-8dc53c70e896 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:14.313858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac717030-e25e-4790-a637-f53473fe319a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:14.314380 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac717030-e25e-4790-a637-f53473fe319a admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.314655 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac717030-e25e-4790-a637-f53473fe319a admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.314921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac717030-e25e-4790-a637-f53473fe319a admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.315578 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac717030-e25e-4790-a637-f53473fe319a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:14.317480 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac717030-e25e-4790-a637-f53473fe319a admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:14.338347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1351/2713] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:14 2022] POST /identity/v3/users => generated 310 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:14.342451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8b50e-121b-49e1-b612-3673ddfa1d4e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:14.353398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd8b50e-121b-49e1-b612-3673ddfa1d4e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:14.358112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e660fe-e029-4460-a770-f102a31c99b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:14.358706 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e660fe-e029-4460-a770-f102a31c99b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.358980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e660fe-e029-4460-a770-f102a31c99b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.359249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e660fe-e029-4460-a770-f102a31c99b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.359914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e660fe-e029-4460-a770-f102a31c99b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:14.361538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e660fe-e029-4460-a770-f102a31c99b9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:14.372521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1363/2714] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:14.376667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac717030-e25e-4790-a637-f53473fe319a admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:14.387323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac717030-e25e-4790-a637-f53473fe319a admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:14.391917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7479516-5f6f-49c7-ab0d-10784f928c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:14.392573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7479516-5f6f-49c7-ab0d-10784f928c72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.392840 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7479516-5f6f-49c7-ab0d-10784f928c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.393110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7479516-5f6f-49c7-ab0d-10784f928c72 admin admin] PATH_INFO: `/v3/projects/5106a69201564e4fab82782aad3f30d0/users/601ca78f0bc84745b4bfe113fe43f76b/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.408261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7479516-5f6f-49c7-ab0d-10784f928c72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5106a69201564e4fab82782aad3f30d0, user_id=601ca78f0bc84745b4bfe113fe43f76b, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:14.410905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7479516-5f6f-49c7-ab0d-10784f928c72 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:14.423825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1352/2715] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:14 2022] PUT /identity/v3/projects/5106a69201564e4fab82782aad3f30d0/users/601ca78f0bc84745b4bfe113fe43f76b/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:14.427993 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e660fe-e029-4460-a770-f102a31c99b9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:14.438277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e660fe-e029-4460-a770-f102a31c99b9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:14.495395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b164f-9639-4e6f-9c42-119c14dce086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:14.496081 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b164f-9639-4e6f-9c42-119c14dce086 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.496299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b164f-9639-4e6f-9c42-119c14dce086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.496566 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b164f-9639-4e6f-9c42-119c14dce086 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.497199 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b164f-9639-4e6f-9c42-119c14dce086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:14.498796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b164f-9639-4e6f-9c42-119c14dce086 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:14.508744 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1364/2716] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:25:14.513024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7479516-5f6f-49c7-ab0d-10784f928c72 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:14.523246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7479516-5f6f-49c7-ab0d-10784f928c72 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:14.528251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b757d2c-005f-40e7-b41b-59b82e6486dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:14.528903 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b757d2c-005f-40e7-b41b-59b82e6486dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.529180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b757d2c-005f-40e7-b41b-59b82e6486dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.529453 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b757d2c-005f-40e7-b41b-59b82e6486dd admin admin] PATH_INFO: `/v3/projects/5106a69201564e4fab82782aad3f30d0/users/601ca78f0bc84745b4bfe113fe43f76b/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.531352 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b757d2c-005f-40e7-b41b-59b82e6486dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5106a69201564e4fab82782aad3f30d0, user_id=601ca78f0bc84745b4bfe113fe43f76b, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:14.534009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b757d2c-005f-40e7-b41b-59b82e6486dd admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:14.544247 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1353/2717] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:14 2022] PUT /identity/v3/projects/5106a69201564e4fab82782aad3f30d0/users/601ca78f0bc84745b4bfe113fe43f76b/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:14.555207 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f62bd-6bcd-4bd5-b816-1422c3ba8148 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.555520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f62bd-6bcd-4bd5-b816-1422c3ba8148 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.555795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629f62bd-6bcd-4bd5-b816-1422c3ba8148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.584208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-629f62bd-6bcd-4bd5-b816-1422c3ba8148 None None] MFA Rules not processed for user `601ca78f0bc84745b4bfe113fe43f76b`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:14.585352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-629f62bd-6bcd-4bd5-b816-1422c3ba8148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:14.601721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:14.655301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1365/2718] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:14 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:14.666021 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b757d2c-005f-40e7-b41b-59b82e6486dd admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:14.668148 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9b757d2c-005f-40e7-b41b-59b82e6486dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:14.714180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:14.732382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b757d2c-005f-40e7-b41b-59b82e6486dd admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:14.735968 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fc613-2b06-478d-ba6c-8d4886543ae3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:14.736569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fc613-2b06-478d-ba6c-8d4886543ae3 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.736845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fc613-2b06-478d-ba6c-8d4886543ae3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.737114 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fc613-2b06-478d-ba6c-8d4886543ae3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.746696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fc613-2b06-478d-ba6c-8d4886543ae3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:14.748154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fc613-2b06-478d-ba6c-8d4886543ae3 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:14.758017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1354/2719] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:14 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:14.912819 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f62bd-6bcd-4bd5-b816-1422c3ba8148 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:14.923906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629f62bd-6bcd-4bd5-b816-1422c3ba8148 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:14.927956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a22439-d24a-48b5-a71a-13f71c86b7f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:14.928552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a22439-d24a-48b5-a71a-13f71c86b7f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:14.928824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a22439-d24a-48b5-a71a-13f71c86b7f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:14.929087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a22439-d24a-48b5-a71a-13f71c86b7f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:14.989378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a22439-d24a-48b5-a71a-13f71c86b7f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:14.990814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a22439-d24a-48b5-a71a-13f71c86b7f5 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:15.000595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1366/2720] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:14 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:24.753266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=10 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:24.996575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=49 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:33.849451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fc613-2b06-478d-ba6c-8d4886543ae3 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:33.921575 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fc613-2b06-478d-ba6c-8d4886543ae3 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:33.926917 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:33.927570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:33.927843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:33.928227 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] PATH_INFO: `/v3/users/68216817a6884853a1a5449d7e675887` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:33.929680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68216817a6884853a1a5449d7e675887)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:33.931671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:33.978448 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68216817a6884853a1a5449d7e675887'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:33.979027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68216817a6884853a1a5449d7e675887'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:33.986176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68216817a6884853a1a5449d7e675887'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:33.994104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68216817a6884853a1a5449d7e675887'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:33.994757 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68216817a6884853a1a5449d7e675887'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.012250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1355/2721] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:33 2022] DELETE /identity/v3/users/68216817a6884853a1a5449d7e675887 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:34.016750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b10ce5-71b3-44fe-9a3c-d86b83095d27 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:34.017253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b10ce5-71b3-44fe-9a3c-d86b83095d27 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:34.017353 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b10ce5-71b3-44fe-9a3c-d86b83095d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:34.035922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-a2b10ce5-71b3-44fe-9a3c-d86b83095d27 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:34.037116 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a2b10ce5-71b3-44fe-9a3c-d86b83095d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:34.095189 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1367/2722] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:34.105028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:34.106454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:34.163640 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c990364-ee83-4dd2-ba58-0c50ee4711b0 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:34.167187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd4b76-abc2-43d6-89d1-efbbbaab2566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:34.167886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd4b76-abc2-43d6-89d1-efbbbaab2566 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:34.168159 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd4b76-abc2-43d6-89d1-efbbbaab2566 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:34.168424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bd4b76-abc2-43d6-89d1-efbbbaab2566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:34.179653 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd4b76-abc2-43d6-89d1-efbbbaab2566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:34.181069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bd4b76-abc2-43d6-89d1-efbbbaab2566 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:34.191877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1356/2723] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:34.463013 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b10ce5-71b3-44fe-9a3c-d86b83095d27 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:34.464532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-a2b10ce5-71b3-44fe-9a3c-d86b83095d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:34.477678 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b10ce5-71b3-44fe-9a3c-d86b83095d27 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:34.482392 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:34.483028 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:34.483313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:34.483641 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] PATH_INFO: `/v3/projects/f6fa2f8b1595409285ca525d27be774d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:34.484764 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6fa2f8b1595409285ca525d27be774d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:34.486688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:34.533645 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6fa2f8b1595409285ca525d27be774d', 'request_id': 'req-3af19aab-5c7c-4be3-a742-733ceb78292e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.539865 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6fa2f8b1595409285ca525d27be774d', 'request_id': 'req-3af19aab-5c7c-4be3-a742-733ceb78292e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.547517 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6fa2f8b1595409285ca525d27be774d', 'request_id': 'req-3af19aab-5c7c-4be3-a742-733ceb78292e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.559395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] The token cache is being invalidate because project f6fa2f8b1595409285ca525d27be774d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:34.559750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.560370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.561840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1368/2724] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:34 2022] DELETE /identity/v3/projects/f6fa2f8b1595409285ca525d27be774d => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:34.566037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd4b76-abc2-43d6-89d1-efbbbaab2566 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:34.575836 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bd4b76-abc2-43d6-89d1-efbbbaab2566 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:34.593674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:34.635414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:34.636066 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:34.636343 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:34.636614 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] PATH_INFO: `/v3/users/601ca78f0bc84745b4bfe113fe43f76b` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:34.637728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=601ca78f0bc84745b4bfe113fe43f76b)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:34.639694 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:34.686418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '601ca78f0bc84745b4bfe113fe43f76b'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.687061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '601ca78f0bc84745b4bfe113fe43f76b'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.694977 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '601ca78f0bc84745b4bfe113fe43f76b'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.701855 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '601ca78f0bc84745b4bfe113fe43f76b'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.702556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-21a56fc1-6de7-4db5-9999-18243fa88fa0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '601ca78f0bc84745b4bfe113fe43f76b'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:34.721107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1357/2725] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:34 2022] DELETE /identity/v3/users/601ca78f0bc84745b4bfe113fe43f76b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:34.999890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.001560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:35.016626 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af19aab-5c7c-4be3-a742-733ceb78292e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.073350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.073992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.074172 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.074444 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] PATH_INFO: `/v3/projects/5106a69201564e4fab82782aad3f30d0` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.075593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5106a69201564e4fab82782aad3f30d0)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.077536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.108140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:35.126309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5106a69201564e4fab82782aad3f30d0', 'request_id': 'req-886866ee-71b6-43df-88ad-9b642413d14e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:35.131788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5106a69201564e4fab82782aad3f30d0', 'request_id': 'req-886866ee-71b6-43df-88ad-9b642413d14e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:35.138873 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5106a69201564e4fab82782aad3f30d0', 'request_id': 'req-886866ee-71b6-43df-88ad-9b642413d14e'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:35.149954 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] The token cache is being invalidate because project 5106a69201564e4fab82782aad3f30d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:35.150241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:35.150883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-886866ee-71b6-43df-88ad-9b642413d14e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:35.152382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1369/2726] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:34 2022] DELETE /identity/v3/projects/5106a69201564e4fab82782aad3f30d0 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:35.157311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70203b11-d0ef-4f35-8c2b-994039d8368f None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.157824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70203b11-d0ef-4f35-8c2b-994039d8368f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.157920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70203b11-d0ef-4f35-8c2b-994039d8368f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.178048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-70203b11-d0ef-4f35-8c2b-994039d8368f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:35.180106 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-70203b11-d0ef-4f35-8c2b-994039d8368f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:35.237601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1358/2727] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:35.246398 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe0fd2-8aab-4e5b-a0b9-2e6872c34f72 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.246752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe0fd2-8aab-4e5b-a0b9-2e6872c34f72 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.247033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbe0fd2-8aab-4e5b-a0b9-2e6872c34f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.266457 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-2dbe0fd2-8aab-4e5b-a0b9-2e6872c34f72 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:35.267666 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2dbe0fd2-8aab-4e5b-a0b9-2e6872c34f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:35.275518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1370/2728] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:35.279656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70203b11-d0ef-4f35-8c2b-994039d8368f None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.290175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70203b11-d0ef-4f35-8c2b-994039d8368f None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.294696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dd7c6-d65a-4aa6-9dd2-a19daf9c2665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.295286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dd7c6-d65a-4aa6-9dd2-a19daf9c2665 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.295610 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dd7c6-d65a-4aa6-9dd2-a19daf9c2665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.295877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26dd7c6-d65a-4aa6-9dd2-a19daf9c2665 admin admin] PATH_INFO: `/v3/domains` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.296510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dd7c6-d65a-4aa6-9dd2-a19daf9c2665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81588) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:25:35.296809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dd7c6-d65a-4aa6-9dd2-a19daf9c2665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.298399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26dd7c6-d65a-4aa6-9dd2-a19daf9c2665 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.310382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1359/2729] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:25:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:25:35.314528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe0fd2-8aab-4e5b-a0b9-2e6872c34f72 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.323718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbe0fd2-8aab-4e5b-a0b9-2e6872c34f72 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.328214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c1873-ef39-4386-9212-ebd0fc265670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.328791 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c1873-ef39-4386-9212-ebd0fc265670 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.329061 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c1873-ef39-4386-9212-ebd0fc265670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.329326 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c1873-ef39-4386-9212-ebd0fc265670 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.329948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c1873-ef39-4386-9212-ebd0fc265670 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.331863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129c1873-ef39-4386-9212-ebd0fc265670 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.361772 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1371/2730] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:35 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:25:35.367067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dd7c6-d65a-4aa6-9dd2-a19daf9c2665 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.376877 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26dd7c6-d65a-4aa6-9dd2-a19daf9c2665 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.403306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:35.437375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e539df-ea90-4703-9348-8d78b880f9cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.437985 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e539df-ea90-4703-9348-8d78b880f9cb admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.438261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e539df-ea90-4703-9348-8d78b880f9cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.438531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e539df-ea90-4703-9348-8d78b880f9cb admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.439180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e539df-ea90-4703-9348-8d78b880f9cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.441132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e539df-ea90-4703-9348-8d78b880f9cb admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.463160 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1360/2731] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:35 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:25:35.467357 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c1873-ef39-4386-9212-ebd0fc265670 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.477934 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c1873-ef39-4386-9212-ebd0fc265670 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.483205 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143530a7-1b54-45ba-bbee-597bfe39ef2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.483833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143530a7-1b54-45ba-bbee-597bfe39ef2c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.484111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143530a7-1b54-45ba-bbee-597bfe39ef2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.484441 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143530a7-1b54-45ba-bbee-597bfe39ef2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.485324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143530a7-1b54-45ba-bbee-597bfe39ef2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.486959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143530a7-1b54-45ba-bbee-597bfe39ef2c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.497868 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1372/2732] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:35.501860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e539df-ea90-4703-9348-8d78b880f9cb admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.512316 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e539df-ea90-4703-9348-8d78b880f9cb admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.516876 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bcdbaa-a9bb-4161-8911-f441cf3fc7f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.517464 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcdbaa-a9bb-4161-8911-f441cf3fc7f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.517741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcdbaa-a9bb-4161-8911-f441cf3fc7f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.518015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bcdbaa-a9bb-4161-8911-f441cf3fc7f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.518647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcdbaa-a9bb-4161-8911-f441cf3fc7f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.520281 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bcdbaa-a9bb-4161-8911-f441cf3fc7f4 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.530592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1361/2733] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:35.534790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143530a7-1b54-45ba-bbee-597bfe39ef2c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.544727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143530a7-1b54-45ba-bbee-597bfe39ef2c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.549542 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a188ad-9724-4bb8-a61b-4f158261eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.550181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a188ad-9724-4bb8-a61b-4f158261eb13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.550458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a188ad-9724-4bb8-a61b-4f158261eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.550736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a188ad-9724-4bb8-a61b-4f158261eb13 admin admin] PATH_INFO: `/v3/projects/74bde919d40c40cdbd550b417614aedd/users/925f8540a88040cabaa19a995ec80913/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.565817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a188ad-9724-4bb8-a61b-4f158261eb13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74bde919d40c40cdbd550b417614aedd, user_id=925f8540a88040cabaa19a995ec80913, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.568521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a188ad-9724-4bb8-a61b-4f158261eb13 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.581741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1373/2734] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:35 2022] PUT /identity/v3/projects/74bde919d40c40cdbd550b417614aedd/users/925f8540a88040cabaa19a995ec80913/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:35.591955 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bca3e-84e8-4274-b813-5c271fd65617 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.592276 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bca3e-84e8-4274-b813-5c271fd65617 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.592553 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186bca3e-84e8-4274-b813-5c271fd65617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.621244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-186bca3e-84e8-4274-b813-5c271fd65617 None None] MFA Rules not processed for user `925f8540a88040cabaa19a995ec80913`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:35.622446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-186bca3e-84e8-4274-b813-5c271fd65617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:35.685659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1362/2735] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:35.690105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a188ad-9724-4bb8-a61b-4f158261eb13 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.752038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a188ad-9724-4bb8-a61b-4f158261eb13 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.756895 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5977b9-2ad4-41f2-91df-bccec4cfa43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.757515 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5977b9-2ad4-41f2-91df-bccec4cfa43d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.757793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5977b9-2ad4-41f2-91df-bccec4cfa43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.758070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5977b9-2ad4-41f2-91df-bccec4cfa43d admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.758698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5977b9-2ad4-41f2-91df-bccec4cfa43d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.760646 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5977b9-2ad4-41f2-91df-bccec4cfa43d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.786537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1374/2736] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:35 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:35.790962 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bca3e-84e8-4274-b813-5c271fd65617 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.801787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186bca3e-84e8-4274-b813-5c271fd65617 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.861323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0345b0-db07-419f-84d6-7c1c37fae093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.861940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0345b0-db07-419f-84d6-7c1c37fae093 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.862215 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0345b0-db07-419f-84d6-7c1c37fae093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.862488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0345b0-db07-419f-84d6-7c1c37fae093 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.863118 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0345b0-db07-419f-84d6-7c1c37fae093 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.865329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0345b0-db07-419f-84d6-7c1c37fae093 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.886826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1363/2737] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:35 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:35.890651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5977b9-2ad4-41f2-91df-bccec4cfa43d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.901448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5977b9-2ad4-41f2-91df-bccec4cfa43d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.906058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222e044-5a61-4632-82f1-8e3e2fad80b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.906647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222e044-5a61-4632-82f1-8e3e2fad80b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.906919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222e044-5a61-4632-82f1-8e3e2fad80b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.907189 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222e044-5a61-4632-82f1-8e3e2fad80b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.907859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222e044-5a61-4632-82f1-8e3e2fad80b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.909445 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222e044-5a61-4632-82f1-8e3e2fad80b6 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.920301 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1375/2738] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:35.924718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0345b0-db07-419f-84d6-7c1c37fae093 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.935869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0345b0-db07-419f-84d6-7c1c37fae093 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.940420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b0771-6181-40d1-a740-435039d1a8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.941093 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b0771-6181-40d1-a740-435039d1a8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.941370 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b0771-6181-40d1-a740-435039d1a8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.941637 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b0771-6181-40d1-a740-435039d1a8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.942303 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b0771-6181-40d1-a740-435039d1a8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.943970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b0771-6181-40d1-a740-435039d1a8e7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:35.953694 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1364/2739] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:25:35.957870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222e044-5a61-4632-82f1-8e3e2fad80b6 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:35.967939 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222e044-5a61-4632-82f1-8e3e2fad80b6 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:35.972529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a7f42-8478-47a2-94dd-957fbe7fa02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:35.973169 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a7f42-8478-47a2-94dd-957fbe7fa02e admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:35.973490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a7f42-8478-47a2-94dd-957fbe7fa02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:35.973766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3a7f42-8478-47a2-94dd-957fbe7fa02e admin admin] PATH_INFO: `/v3/projects/98d6cbe84adb4f3aabe138d7b2583ebd/users/bd2232d96dba4b4fa2d0fa920be33e43/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:35.989388 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a7f42-8478-47a2-94dd-957fbe7fa02e admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d6cbe84adb4f3aabe138d7b2583ebd, user_id=bd2232d96dba4b4fa2d0fa920be33e43, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:35.993095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3a7f42-8478-47a2-94dd-957fbe7fa02e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:36.003355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:36.009141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1376/2740] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:35 2022] PUT /identity/v3/projects/98d6cbe84adb4f3aabe138d7b2583ebd/users/bd2232d96dba4b4fa2d0fa920be33e43/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:36.083851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f81d0-50d4-4e38-b829-3cc57bca39a4 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:36.084132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f81d0-50d4-4e38-b829-3cc57bca39a4 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:36.084451 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007f81d0-50d4-4e38-b829-3cc57bca39a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:36.112994 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-007f81d0-50d4-4e38-b829-3cc57bca39a4 None None] MFA Rules not processed for user `bd2232d96dba4b4fa2d0fa920be33e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:36.114397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-007f81d0-50d4-4e38-b829-3cc57bca39a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:36.119282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:36.181162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1365/2741] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:36.185661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a7f42-8478-47a2-94dd-957fbe7fa02e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:36.246263 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3a7f42-8478-47a2-94dd-957fbe7fa02e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:36.250894 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1cffc-0968-4e7b-b5d5-2e6e32446768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:36.251625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1cffc-0968-4e7b-b5d5-2e6e32446768 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:36.251904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1cffc-0968-4e7b-b5d5-2e6e32446768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:36.252176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1cffc-0968-4e7b-b5d5-2e6e32446768 admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:36.252792 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1cffc-0968-4e7b-b5d5-2e6e32446768 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:36.254695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1cffc-0968-4e7b-b5d5-2e6e32446768 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:36.282380 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1377/2742] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:36 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:36.286442 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f81d0-50d4-4e38-b829-3cc57bca39a4 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:36.297763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007f81d0-50d4-4e38-b829-3cc57bca39a4 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:36.353125 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d97af-ce2a-4092-bea3-5976445fde50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:36.353725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d97af-ce2a-4092-bea3-5976445fde50 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:36.354003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d97af-ce2a-4092-bea3-5976445fde50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:36.354274 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d97af-ce2a-4092-bea3-5976445fde50 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:36.354895 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d97af-ce2a-4092-bea3-5976445fde50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:36.356856 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d97af-ce2a-4092-bea3-5976445fde50 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:36.377794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1366/2743] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:36 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:36.381818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1cffc-0968-4e7b-b5d5-2e6e32446768 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:36.391950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1cffc-0968-4e7b-b5d5-2e6e32446768 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:36.397435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f4fa0-3802-4829-9bb7-f6083c6a0ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:36.398243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4fa0-3802-4829-9bb7-f6083c6a0ffa admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:36.398637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4fa0-3802-4829-9bb7-f6083c6a0ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:36.398912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6f4fa0-3802-4829-9bb7-f6083c6a0ffa admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:36.399615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f4fa0-3802-4829-9bb7-f6083c6a0ffa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:36.401196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6f4fa0-3802-4829-9bb7-f6083c6a0ffa admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:36.411738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1378/2744] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:36.417026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d97af-ce2a-4092-bea3-5976445fde50 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:36.427279 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d97af-ce2a-4092-bea3-5976445fde50 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:36.444929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28addf-0d58-42dc-9c5e-998159f2f218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:36.444929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28addf-0d58-42dc-9c5e-998159f2f218 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:36.444929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28addf-0d58-42dc-9c5e-998159f2f218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:36.444929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28addf-0d58-42dc-9c5e-998159f2f218 admin admin] PATH_INFO: `/v3/projects/060729b788de40cc8d4d5b8d14db3e23/users/9931110827a2429b9b600200e8adda98/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:36.451305 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28addf-0d58-42dc-9c5e-998159f2f218 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060729b788de40cc8d4d5b8d14db3e23, user_id=9931110827a2429b9b600200e8adda98, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:36.454011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28addf-0d58-42dc-9c5e-998159f2f218 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:36.467665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1367/2745] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:36 2022] PUT /identity/v3/projects/060729b788de40cc8d4d5b8d14db3e23/users/9931110827a2429b9b600200e8adda98/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:25:36.471720 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f4fa0-3802-4829-9bb7-f6083c6a0ffa admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:36.482266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6f4fa0-3802-4829-9bb7-f6083c6a0ffa admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:36.536609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd30d07c-660c-4cd5-b10a-fdbebfd00628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:36.537192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30d07c-660c-4cd5-b10a-fdbebfd00628 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:36.538490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30d07c-660c-4cd5-b10a-fdbebfd00628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:36.538766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd30d07c-660c-4cd5-b10a-fdbebfd00628 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:36.539396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30d07c-660c-4cd5-b10a-fdbebfd00628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:36.541014 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd30d07c-660c-4cd5-b10a-fdbebfd00628 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:36.550456 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1379/2746] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:36.554602 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28addf-0d58-42dc-9c5e-998159f2f218 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:36.565026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28addf-0d58-42dc-9c5e-998159f2f218 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:36.570539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c05b8-1219-408f-9d45-40f48cf466f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:36.571186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c05b8-1219-408f-9d45-40f48cf466f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:36.571529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c05b8-1219-408f-9d45-40f48cf466f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:36.571809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9c05b8-1219-408f-9d45-40f48cf466f7 admin admin] PATH_INFO: `/v3/projects/060729b788de40cc8d4d5b8d14db3e23/users/9931110827a2429b9b600200e8adda98/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:36.573753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c05b8-1219-408f-9d45-40f48cf466f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=060729b788de40cc8d4d5b8d14db3e23, user_id=9931110827a2429b9b600200e8adda98, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:36.576485 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9c05b8-1219-408f-9d45-40f48cf466f7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:36.587239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1368/2747] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:36 2022] PUT /identity/v3/projects/060729b788de40cc8d4d5b8d14db3e23/users/9931110827a2429b9b600200e8adda98/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:36.597025 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d27ca-d476-40f4-a114-4f23308d4216 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:36.597306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d27ca-d476-40f4-a114-4f23308d4216 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:36.597576 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d27ca-d476-40f4-a114-4f23308d4216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:36.625127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-ba6d27ca-d476-40f4-a114-4f23308d4216 None None] MFA Rules not processed for user `9931110827a2429b9b600200e8adda98`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:36.626329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ba6d27ca-d476-40f4-a114-4f23308d4216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:36.685242 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:36.695708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1380/2748] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:25:36.708028 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c05b8-1219-408f-9d45-40f48cf466f7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:36.710327 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-aa9c05b8-1219-408f-9d45-40f48cf466f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:36.766364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9c05b8-1219-408f-9d45-40f48cf466f7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:36.770017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e0d34-7331-41c7-adce-f4217e5eb3fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:36.770608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e0d34-7331-41c7-adce-f4217e5eb3fb service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:36.770884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e0d34-7331-41c7-adce-f4217e5eb3fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:36.771152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e0d34-7331-41c7-adce-f4217e5eb3fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:36.780921 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:36.828773 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e0d34-7331-41c7-adce-f4217e5eb3fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:36.830206 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e0d34-7331-41c7-adce-f4217e5eb3fb service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:36.839681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1369/2749] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:39.404679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d27ca-d476-40f4-a114-4f23308d4216 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:39.415581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d27ca-d476-40f4-a114-4f23308d4216 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:39.419818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d93223-0118-495d-8f33-81a150f73bb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:39.420498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93223-0118-495d-8f33-81a150f73bb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:39.420832 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93223-0118-495d-8f33-81a150f73bb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:39.421161 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d93223-0118-495d-8f33-81a150f73bb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:39.490831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d93223-0118-495d-8f33-81a150f73bb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:39.492348 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d93223-0118-495d-8f33-81a150f73bb8 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:39.505177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1381/2750] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:40.061042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e0d34-7331-41c7-adce-f4217e5eb3fb service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:40.069746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e0d34-7331-41c7-adce-f4217e5eb3fb service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:40.073391 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e92803-30b9-452a-844c-041e432c1253 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:40.073987 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e92803-30b9-452a-844c-041e432c1253 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:40.074263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e92803-30b9-452a-844c-041e432c1253 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:40.074531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e92803-30b9-452a-844c-041e432c1253 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:40.083304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e92803-30b9-452a-844c-041e432c1253 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:40.085295 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e92803-30b9-452a-844c-041e432c1253 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:40.094591 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1370/2751] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:46.651194 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d93223-0118-495d-8f33-81a150f73bb8 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:46.652844 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c8d93223-0118-495d-8f33-81a150f73bb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:46.728183 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d93223-0118-495d-8f33-81a150f73bb8 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:46.732825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:46.733459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:46.733737 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:46.734012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] PATH_INFO: `/v3/users/1c100121ca05424ebf9f470b89f8acbb` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:46.735171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c100121ca05424ebf9f470b89f8acbb)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:46.737176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:46.788108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c100121ca05424ebf9f470b89f8acbb'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:46.797474 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c100121ca05424ebf9f470b89f8acbb'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:46.798109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c100121ca05424ebf9f470b89f8acbb'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:46.798729 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c100121ca05424ebf9f470b89f8acbb'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:46.806594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c100121ca05424ebf9f470b89f8acbb'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:46.824109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1382/2752] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:46 2022] DELETE /identity/v3/users/1c100121ca05424ebf9f470b89f8acbb => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:25:46.828697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d746-b8e4-4e41-bbe0-d14cf6ea829a None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:46.829180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d746-b8e4-4e41-bbe0-d14cf6ea829a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:46.829297 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210d746-b8e4-4e41-bbe0-d14cf6ea829a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:46.849030 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-5210d746-b8e4-4e41-bbe0-d14cf6ea829a None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:46.850063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5210d746-b8e4-4e41-bbe0-d14cf6ea829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:46.910376 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1371/2753] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:25:46.920443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:46.921871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:46.938002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:46.985438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310c104-2c80-4bb3-abc1-9fd132715aaa admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:46.990043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671ae47-7266-4326-b0dd-4b8d13f17222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:46.990661 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671ae47-7266-4326-b0dd-4b8d13f17222 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:46.991042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671ae47-7266-4326-b0dd-4b8d13f17222 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:46.991938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671ae47-7266-4326-b0dd-4b8d13f17222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:47.001480 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671ae47-7266-4326-b0dd-4b8d13f17222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:47.002979 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671ae47-7266-4326-b0dd-4b8d13f17222 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:47.013745 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1383/2754] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:47.148819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210d746-b8e4-4e41-bbe0-d14cf6ea829a None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:47.150242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5210d746-b8e4-4e41-bbe0-d14cf6ea829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:47.162282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210d746-b8e4-4e41-bbe0-d14cf6ea829a None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:47.168669 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:47.169387 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:47.169734 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:47.170069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] PATH_INFO: `/v3/users/925f8540a88040cabaa19a995ec80913` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:47.171312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] RBAC: Authorizing `identity:delete_user(user_id=925f8540a88040cabaa19a995ec80913)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:47.173462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:47.233737 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '925f8540a88040cabaa19a995ec80913'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.234427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '925f8540a88040cabaa19a995ec80913'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.244113 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '925f8540a88040cabaa19a995ec80913'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.251989 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '925f8540a88040cabaa19a995ec80913'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.252667 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '925f8540a88040cabaa19a995ec80913'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.276077 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1372/2755] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:47 2022] DELETE /identity/v3/users/925f8540a88040cabaa19a995ec80913 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:47.278300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b814a-4912-4a44-8b6b-905914a32a81 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:47.278659 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b814a-4912-4a44-8b6b-905914a32a81 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:47.279007 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87b814a-4912-4a44-8b6b-905914a32a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:47.300405 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-e87b814a-4912-4a44-8b6b-905914a32a81 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:47.301826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e87b814a-4912-4a44-8b6b-905914a32a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:47.308564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:47.310231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:47.327147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:47.386444 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1384/2756] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:47.388607 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b814a-4912-4a44-8b6b-905914a32a81 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:47.390170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e87b814a-4912-4a44-8b6b-905914a32a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:47.394754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afddb4-5816-4cf7-afbe-197330a50859 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:47.399943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:47.400579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:47.400858 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:47.401128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] PATH_INFO: `/v3/projects/49973800fe3b4cccbf9ee95361330830` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:47.402296 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49973800fe3b4cccbf9ee95361330830)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:47.403173 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87b814a-4912-4a44-8b6b-905914a32a81 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:47.404251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:47.407937 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:47.408557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:47.408974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:47.409257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] PATH_INFO: `/v3/users/7dcf25bcabb342cf895333fce7e422ef` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:47.410422 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dcf25bcabb342cf895333fce7e422ef)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:47.412375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:47.456763 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49973800fe3b4cccbf9ee95361330830', 'request_id': 'req-377b9a0b-828c-4a4b-a548-ddc38731f8c0'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.464812 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49973800fe3b4cccbf9ee95361330830', 'request_id': 'req-377b9a0b-828c-4a4b-a548-ddc38731f8c0'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.470133 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dcf25bcabb342cf895333fce7e422ef'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.470909 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49973800fe3b4cccbf9ee95361330830', 'request_id': 'req-377b9a0b-828c-4a4b-a548-ddc38731f8c0'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.478721 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dcf25bcabb342cf895333fce7e422ef'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.479424 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dcf25bcabb342cf895333fce7e422ef'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.480187 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dcf25bcabb342cf895333fce7e422ef'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.483718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] The token cache is being invalidate because project 49973800fe3b4cccbf9ee95361330830 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:47.484083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.484800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.486412 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1373/2757] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:47 2022] DELETE /identity/v3/projects/49973800fe3b4cccbf9ee95361330830 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:47.487230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:47.487378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dcf25bcabb342cf895333fce7e422ef'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.488904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:47.506694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1385/2758] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:47 2022] DELETE /identity/v3/users/7dcf25bcabb342cf895333fce7e422ef => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:47.507606 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:47.508925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:47.542047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:47.550733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377b9a0b-828c-4a4b-a548-ddc38731f8c0 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:47.577899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063df417-b31f-4049-baaa-998bb1fe5a33 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:47.583789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:47.584585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:47.584959 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:47.585870 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] PATH_INFO: `/v3/users/531c382338974251ab773eaf9d9bad8a` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:47.587254 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=531c382338974251ab773eaf9d9bad8a)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:47.589653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:47.605603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c5d610-6f10-4bcf-b370-7b9d06420c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:47.606232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5d610-6f10-4bcf-b370-7b9d06420c1d service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:47.606694 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5d610-6f10-4bcf-b370-7b9d06420c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:47.607119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c5d610-6f10-4bcf-b370-7b9d06420c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:47.609084 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-12c5d610-6f10-4bcf-b370-7b9d06420c1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:47.626919 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5d610-6f10-4bcf-b370-7b9d06420c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:47.628413 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c5d610-6f10-4bcf-b370-7b9d06420c1d service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:47.635190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:47.640741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1374/2759] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:47.642072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac0a16-59c7-464e-99bc-5be2f73d3643 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:47.642728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac0a16-59c7-464e-99bc-5be2f73d3643 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:47.643074 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac0a16-59c7-464e-99bc-5be2f73d3643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:47.646299 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '531c382338974251ab773eaf9d9bad8a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.659846 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '531c382338974251ab773eaf9d9bad8a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.660596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '531c382338974251ab773eaf9d9bad8a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.661193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '531c382338974251ab773eaf9d9bad8a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.665611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-f9ac0a16-59c7-464e-99bc-5be2f73d3643 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:47.666916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f9ac0a16-59c7-464e-99bc-5be2f73d3643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:47.669214 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '531c382338974251ab773eaf9d9bad8a'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:47.675314 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1375/2760] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:47.693069 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac0a16-59c7-464e-99bc-5be2f73d3643 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:47.698479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1386/2761] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:47 2022] DELETE /identity/v3/users/531c382338974251ab773eaf9d9bad8a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:47.698803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f9ac0a16-59c7-464e-99bc-5be2f73d3643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:47.716359 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac0a16-59c7-464e-99bc-5be2f73d3643 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:47.778823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182cb00-0e71-49a4-9696-7a42bcee1b87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:47.779659 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182cb00-0e71-49a4-9696-7a42bcee1b87 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:47.780058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182cb00-0e71-49a4-9696-7a42bcee1b87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:47.780447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182cb00-0e71-49a4-9696-7a42bcee1b87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:47.850479 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182cb00-0e71-49a4-9696-7a42bcee1b87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:47.852692 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182cb00-0e71-49a4-9696-7a42bcee1b87 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:47.863270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1376/2762] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:48.084712 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.087067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:48.101604 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0f9c99-5c4a-4f6f-83be-361a4a91c8f2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.116355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.117095 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.117375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.117647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] PATH_INFO: `/v3/projects/74bde919d40c40cdbd550b417614aedd` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.120229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74bde919d40c40cdbd550b417614aedd)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.122473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.177332 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182cb00-0e71-49a4-9696-7a42bcee1b87 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.179246 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5182cb00-0e71-49a4-9696-7a42bcee1b87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:48.186125 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74bde919d40c40cdbd550b417614aedd', 'request_id': 'req-177aa570-a9cf-43da-9754-64299bc99217'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.192767 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74bde919d40c40cdbd550b417614aedd', 'request_id': 'req-177aa570-a9cf-43da-9754-64299bc99217'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.197087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182cb00-0e71-49a4-9696-7a42bcee1b87 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.213782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74bde919d40c40cdbd550b417614aedd', 'request_id': 'req-177aa570-a9cf-43da-9754-64299bc99217'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.228476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] The token cache is being invalidate because project 74bde919d40c40cdbd550b417614aedd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:48.228731 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.229396 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.231588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1387/2763] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:48 2022] DELETE /identity/v3/projects/74bde919d40c40cdbd550b417614aedd => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:48.232483 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.233897 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:48.270876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:48.276436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.277176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.277525 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.277861 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] PATH_INFO: `/v3/projects/39b34c6327d04518aecadc8dae713270` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.279055 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=39b34c6327d04518aecadc8dae713270)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.281138 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.305671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177aa570-a9cf-43da-9754-64299bc99217 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.310881 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.311553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.311809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.312078 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] PATH_INFO: `/v3/projects/599ffb54b5994e428b84adfd9c38481b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.313307 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] RBAC: Authorizing `identity:delete_project(project_id=599ffb54b5994e428b84adfd9c38481b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.315273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.328796 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:48.332173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '39b34c6327d04518aecadc8dae713270', 'request_id': 'req-8fb6ab91-4375-488a-bde9-26b88a759e8d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.340853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '39b34c6327d04518aecadc8dae713270', 'request_id': 'req-8fb6ab91-4375-488a-bde9-26b88a759e8d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.346706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '39b34c6327d04518aecadc8dae713270', 'request_id': 'req-8fb6ab91-4375-488a-bde9-26b88a759e8d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.358611 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] The token cache is being invalidate because project 39b34c6327d04518aecadc8dae713270 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:48.358899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.359548 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.361047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1377/2764] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:48 2022] DELETE /identity/v3/projects/39b34c6327d04518aecadc8dae713270 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:25:48.365768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.365768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:48.365973 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '599ffb54b5994e428b84adfd9c38481b', 'request_id': 'req-1916ccf0-30ca-4446-ab59-50d4ae1c810d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.372295 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '599ffb54b5994e428b84adfd9c38481b', 'request_id': 'req-1916ccf0-30ca-4446-ab59-50d4ae1c810d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.380572 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '599ffb54b5994e428b84adfd9c38481b', 'request_id': 'req-1916ccf0-30ca-4446-ab59-50d4ae1c810d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.392084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] The token cache is being invalidate because project 599ffb54b5994e428b84adfd9c38481b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:48.392367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.393010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-1916ccf0-30ca-4446-ab59-50d4ae1c810d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.394496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1388/2765] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:48 2022] DELETE /identity/v3/projects/599ffb54b5994e428b84adfd9c38481b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:48.395842 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d9b30-1a38-45da-90c5-273888908628 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.396122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d9b30-1a38-45da-90c5-273888908628 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.396391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0d9b30-1a38-45da-90c5-273888908628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.416688 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-fa0d9b30-1a38-45da-90c5-273888908628 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:48.417876 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-fa0d9b30-1a38-45da-90c5-273888908628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:48.434213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb6ab91-4375-488a-bde9-26b88a759e8d admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.478185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1389/2766] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:48.479529 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38ca0d-183a-4ea2-a9bf-c96db1d1f8a6 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.479858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38ca0d-183a-4ea2-a9bf-c96db1d1f8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.480132 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d38ca0d-183a-4ea2-a9bf-c96db1d1f8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.501190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.501695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1d38ca0d-183a-4ea2-a9bf-c96db1d1f8a6 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:48.501850 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.502089 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.502364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] PATH_INFO: `/v3/users/bd2232d96dba4b4fa2d0fa920be33e43` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.502952 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1d38ca0d-183a-4ea2-a9bf-c96db1d1f8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:48.503835 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd2232d96dba4b4fa2d0fa920be33e43)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.505942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.512259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1390/2767] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:48.513479 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5582c4-a026-4fc7-9403-765c6beb22ed None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.513983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5582c4-a026-4fc7-9403-765c6beb22ed None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.514265 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5582c4-a026-4fc7-9403-765c6beb22ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.546886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-af5582c4-a026-4fc7-9403-765c6beb22ed None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:48.548088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-af5582c4-a026-4fc7-9403-765c6beb22ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:48.557593 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1391/2768] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:48.562059 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5582c4-a026-4fc7-9403-765c6beb22ed None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.567520 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd2232d96dba4b4fa2d0fa920be33e43'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.569112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd2232d96dba4b4fa2d0fa920be33e43'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.572122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5582c4-a026-4fc7-9403-765c6beb22ed None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.578345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89a1989-c030-48a2-b6a7-287ccec7abb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.578706 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd2232d96dba4b4fa2d0fa920be33e43'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.579326 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a1989-c030-48a2-b6a7-287ccec7abb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.579652 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a1989-c030-48a2-b6a7-287ccec7abb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.580098 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89a1989-c030-48a2-b6a7-287ccec7abb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.581223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a1989-c030-48a2-b6a7-287ccec7abb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:25:48.581544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a1989-c030-48a2-b6a7-287ccec7abb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.583374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89a1989-c030-48a2-b6a7-287ccec7abb9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.586382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd2232d96dba4b4fa2d0fa920be33e43'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.587060 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd2232d96dba4b4fa2d0fa920be33e43'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:48.599996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1392/2769] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:25:48.601770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:48.601917 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36af62-6caa-4cce-8e8c-a90f0ed4bc47 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.602096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36af62-6caa-4cce-8e8c-a90f0ed4bc47 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.602374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df36af62-6caa-4cce-8e8c-a90f0ed4bc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.607015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1378/2770] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:48 2022] DELETE /identity/v3/users/bd2232d96dba4b4fa2d0fa920be33e43 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:48.608348 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.610007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:48.626860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-df36af62-6caa-4cce-8e8c-a90f0ed4bc47 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:48.628124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-df36af62-6caa-4cce-8e8c-a90f0ed4bc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:48.671584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:48.702618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb958814-3539-4e29-be4b-4d17e4efa67e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.709652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c766d-2856-4cda-a8eb-674817d3dd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.710323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c766d-2856-4cda-a8eb-674817d3dd76 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.710775 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c766d-2856-4cda-a8eb-674817d3dd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.710981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38c766d-2856-4cda-a8eb-674817d3dd76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.711991 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c766d-2856-4cda-a8eb-674817d3dd76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.713671 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1393/2771] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:48.714208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38c766d-2856-4cda-a8eb-674817d3dd76 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.717947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36af62-6caa-4cce-8e8c-a90f0ed4bc47 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.729761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df36af62-6caa-4cce-8e8c-a90f0ed4bc47 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.735130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905edd25-7a2e-4a76-a6d2-df817877bf0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.735854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905edd25-7a2e-4a76-a6d2-df817877bf0a admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.736136 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905edd25-7a2e-4a76-a6d2-df817877bf0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.736411 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905edd25-7a2e-4a76-a6d2-df817877bf0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.737096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905edd25-7a2e-4a76-a6d2-df817877bf0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:25:48.737375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905edd25-7a2e-4a76-a6d2-df817877bf0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.739863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905edd25-7a2e-4a76-a6d2-df817877bf0a admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.746090 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1379/2772] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:48 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:48.750120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c766d-2856-4cda-a8eb-674817d3dd76 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.759542 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1394/2773] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:25:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:25:48.760930 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905edd25-7a2e-4a76-a6d2-df817877bf0a admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.777002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905edd25-7a2e-4a76-a6d2-df817877bf0a admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.833880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38c766d-2856-4cda-a8eb-674817d3dd76 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.839699 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315ab08-ffd2-43d6-b99e-0a9022eee2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.840311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315ab08-ffd2-43d6-b99e-0a9022eee2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.840931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315ab08-ffd2-43d6-b99e-0a9022eee2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.840931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2315ab08-ffd2-43d6-b99e-0a9022eee2a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.841476 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315ab08-ffd2-43d6-b99e-0a9022eee2a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.842535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea63779-40f8-4230-a152-4c3b0f85319f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.843276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea63779-40f8-4230-a152-4c3b0f85319f admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.843429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2315ab08-ffd2-43d6-b99e-0a9022eee2a8 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.844035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea63779-40f8-4230-a152-4c3b0f85319f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.844035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea63779-40f8-4230-a152-4c3b0f85319f admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.844613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea63779-40f8-4230-a152-4c3b0f85319f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.846524 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea63779-40f8-4230-a152-4c3b0f85319f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.871564 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1380/2774] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:48 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:25:48.872454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315ab08-ffd2-43d6-b99e-0a9022eee2a8 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.881212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1395/2775] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:48 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:25:48.884062 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2315ab08-ffd2-43d6-b99e-0a9022eee2a8 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.885819 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea63779-40f8-4230-a152-4c3b0f85319f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.955964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11573-a8e2-40d1-8bf6-34d1083d55a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.956726 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11573-a8e2-40d1-8bf6-34d1083d55a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.957011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11573-a8e2-40d1-8bf6-34d1083d55a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.957325 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f11573-a8e2-40d1-8bf6-34d1083d55a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.957938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea63779-40f8-4230-a152-4c3b0f85319f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.958045 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f11573-a8e2-40d1-8bf6-34d1083d55a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.959526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f11573-a8e2-40d1-8bf6-34d1083d55a7 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.962757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c5d1d-0f32-44e5-b0e2-d55b22f3015c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.963355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c5d1d-0f32-44e5-b0e2-d55b22f3015c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.963656 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c5d1d-0f32-44e5-b0e2-d55b22f3015c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.963926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c5d1d-0f32-44e5-b0e2-d55b22f3015c admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.964545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c5d1d-0f32-44e5-b0e2-d55b22f3015c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.966473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c5d1d-0f32-44e5-b0e2-d55b22f3015c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:48.973250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1381/2776] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:48.973250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11573-a8e2-40d1-8bf6-34d1083d55a7 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.973250 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-03f11573-a8e2-40d1-8bf6-34d1083d55a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:48.986077 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f11573-a8e2-40d1-8bf6-34d1083d55a7 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:48.988476 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1396/2777] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:48 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:48.989099 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c5d1d-0f32-44e5-b0e2-d55b22f3015c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:48.991864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:48.992872 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:48.993162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:48.993424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] PATH_INFO: `/v3/projects/98d6cbe84adb4f3aabe138d7b2583ebd` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:48.994560 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d6cbe84adb4f3aabe138d7b2583ebd)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:48.996502 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.001020 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c5d1d-0f32-44e5-b0e2-d55b22f3015c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.003760 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:49.005627 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b8667-f323-4179-8991-475984fe14e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.006241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b8667-f323-4179-8991-475984fe14e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.006594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b8667-f323-4179-8991-475984fe14e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.006742 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b8667-f323-4179-8991-475984fe14e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.008157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b8667-f323-4179-8991-475984fe14e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.009748 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b8667-f323-4179-8991-475984fe14e4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.019942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1397/2778] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:49.021077 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b8667-f323-4179-8991-475984fe14e4 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.031805 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b8667-f323-4179-8991-475984fe14e4 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.036658 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1165950-c203-44e5-b107-803fe93f2018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.037234 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1165950-c203-44e5-b107-803fe93f2018 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.037507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1165950-c203-44e5-b107-803fe93f2018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.037774 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1165950-c203-44e5-b107-803fe93f2018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.038602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1165950-c203-44e5-b107-803fe93f2018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.040419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1165950-c203-44e5-b107-803fe93f2018 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.049129 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d6cbe84adb4f3aabe138d7b2583ebd', 'request_id': 'req-055db0b2-b7d4-4a73-a183-080646a0a2b2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.050853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1398/2779] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:49.050853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1165950-c203-44e5-b107-803fe93f2018 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.060344 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d6cbe84adb4f3aabe138d7b2583ebd', 'request_id': 'req-055db0b2-b7d4-4a73-a183-080646a0a2b2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.061919 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1165950-c203-44e5-b107-803fe93f2018 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.066162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d6cbe84adb4f3aabe138d7b2583ebd', 'request_id': 'req-055db0b2-b7d4-4a73-a183-080646a0a2b2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.073856 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:49.078818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] The token cache is being invalidate because project 98d6cbe84adb4f3aabe138d7b2583ebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:49.079110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.079860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.084307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1382/2780] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:48 2022] DELETE /identity/v3/projects/98d6cbe84adb4f3aabe138d7b2583ebd => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:49.084307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.084307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.123586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565f0a2-7852-4001-80e0-6efc4352e8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.124362 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565f0a2-7852-4001-80e0-6efc4352e8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.124642 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565f0a2-7852-4001-80e0-6efc4352e8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.124922 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565f0a2-7852-4001-80e0-6efc4352e8bc admin admin] PATH_INFO: `/v3/projects/4cdf4f2f64fc4b91b144e1ea031115a1/users/f080ef0e6d0b46b5876316042e99d910/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.139893 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565f0a2-7852-4001-80e0-6efc4352e8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cdf4f2f64fc4b91b144e1ea031115a1, user_id=f080ef0e6d0b46b5876316042e99d910, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.142568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565f0a2-7852-4001-80e0-6efc4352e8bc admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.149866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055db0b2-b7d4-4a73-a183-080646a0a2b2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.155825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1399/2781] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:49 2022] PUT /identity/v3/projects/4cdf4f2f64fc4b91b144e1ea031115a1/users/f080ef0e6d0b46b5876316042e99d910/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:49.156413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565f0a2-7852-4001-80e0-6efc4352e8bc admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.166255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0565f0a2-7852-4001-80e0-6efc4352e8bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.213459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef09c05-5cdf-4695-bf21-0302d2cc250c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.214072 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef09c05-5cdf-4695-bf21-0302d2cc250c admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.214349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef09c05-5cdf-4695-bf21-0302d2cc250c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.214623 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef09c05-5cdf-4695-bf21-0302d2cc250c admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.215262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef09c05-5cdf-4695-bf21-0302d2cc250c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.216896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef09c05-5cdf-4695-bf21-0302d2cc250c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.227085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1383/2782] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:49.228662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b58af-9609-4c9e-9319-cd567c197cc6 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.228950 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b58af-9609-4c9e-9319-cd567c197cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.229222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b58af-9609-4c9e-9319-cd567c197cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.234761 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565f0a2-7852-4001-80e0-6efc4352e8bc admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.239807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.240436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.240715 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.240986 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] PATH_INFO: `/v3/users/9931110827a2429b9b600200e8adda98` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.242752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9931110827a2429b9b600200e8adda98)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.244701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.258212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-991b58af-9609-4c9e-9319-cd567c197cc6 None None] MFA Rules not processed for user `f080ef0e6d0b46b5876316042e99d910`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:49.259432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-991b58af-9609-4c9e-9319-cd567c197cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.287470 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:49.292543 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9931110827a2429b9b600200e8adda98'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.301272 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9931110827a2429b9b600200e8adda98'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.302027 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9931110827a2429b9b600200e8adda98'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.302836 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9931110827a2429b9b600200e8adda98'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.310274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9931110827a2429b9b600200e8adda98'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.327904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1400/2783] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:25:49 2022] DELETE /identity/v3/users/9931110827a2429b9b600200e8adda98 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:49.328615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.329100 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1384/2784] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:49.330152 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.335232 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b58af-9609-4c9e-9319-cd567c197cc6 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.336733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-991b58af-9609-4c9e-9319-cd567c197cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.349875 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:49.350226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b58af-9609-4c9e-9319-cd567c197cc6 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.399091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765b500-171f-4c61-b0e7-be9ccdc227d8 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.407200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac78b9b-846d-4fd7-b828-0e6e256a611f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.408220 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac78b9b-846d-4fd7-b828-0e6e256a611f admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.408504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac78b9b-846d-4fd7-b828-0e6e256a611f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.409041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac78b9b-846d-4fd7-b828-0e6e256a611f admin admin] PATH_INFO: `/v3/projects/b9d256f4805f4d45af9a4df1d37f4246/users/c73a4aa58ec848258c4b47d9aad51a0b/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.421162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99cff6f-5d61-4074-8dfb-d095921d6e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.421884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cff6f-5d61-4074-8dfb-d095921d6e63 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.422084 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cff6f-5d61-4074-8dfb-d095921d6e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.422592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99cff6f-5d61-4074-8dfb-d095921d6e63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.423180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cff6f-5d61-4074-8dfb-d095921d6e63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.425233 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99cff6f-5d61-4074-8dfb-d095921d6e63 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.432585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac78b9b-846d-4fd7-b828-0e6e256a611f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d256f4805f4d45af9a4df1d37f4246, user_id=c73a4aa58ec848258c4b47d9aad51a0b, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.437770 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac78b9b-846d-4fd7-b828-0e6e256a611f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.458544 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1401/2785] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:49 2022] PUT /identity/v3/projects/b9d256f4805f4d45af9a4df1d37f4246/users/c73a4aa58ec848258c4b47d9aad51a0b/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:49.460648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1385/2786] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:49 2022] POST /identity/v3/projects => generated 377 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:25:49.464141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac78b9b-846d-4fd7-b828-0e6e256a611f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.469296 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e766b5-9430-41af-b7ea-94f24e143e1e None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.469924 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e766b5-9430-41af-b7ea-94f24e143e1e None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.470954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e766b5-9430-41af-b7ea-94f24e143e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.510031 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-39e766b5-9430-41af-b7ea-94f24e143e1e None None] MFA Rules not processed for user `c73a4aa58ec848258c4b47d9aad51a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:49.511262 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-39e766b5-9430-41af-b7ea-94f24e143e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.550705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac78b9b-846d-4fd7-b828-0e6e256a611f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.589791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1386/2787] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:25:49.601855 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e766b5-9430-41af-b7ea-94f24e143e1e None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.603598 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-39e766b5-9430-41af-b7ea-94f24e143e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.614351 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dbb78-4a3e-4763-a8fb-21f8846452c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.614960 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dbb78-4a3e-4763-a8fb-21f8846452c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.615237 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dbb78-4a3e-4763-a8fb-21f8846452c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.615552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9dbb78-4a3e-4763-a8fb-21f8846452c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.616530 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9dbb78-4a3e-4763-a8fb-21f8846452c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.619016 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9dbb78-4a3e-4763-a8fb-21f8846452c5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.646520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1402/2788] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:49 2022] POST /identity/v3/users => generated 295 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:25:49.648309 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dbb78-4a3e-4763-a8fb-21f8846452c5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.661910 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9dbb78-4a3e-4763-a8fb-21f8846452c5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.668521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ab4e7-4d85-4132-aa6c-7b4a271dc3e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.669312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:49.669451 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ab4e7-4d85-4132-aa6c-7b4a271dc3e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.669624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ab4e7-4d85-4132-aa6c-7b4a271dc3e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.669874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751ab4e7-4d85-4132-aa6c-7b4a271dc3e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.670987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ab4e7-4d85-4132-aa6c-7b4a271dc3e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.673070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751ab4e7-4d85-4132-aa6c-7b4a271dc3e5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.683032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e766b5-9430-41af-b7ea-94f24e143e1e None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.687656 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e62e6-3ee8-4f52-906b-bb1274c38b75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.688178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e62e6-3ee8-4f52-906b-bb1274c38b75 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.688616 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e62e6-3ee8-4f52-906b-bb1274c38b75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.688741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657e62e6-3ee8-4f52-906b-bb1274c38b75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.696257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1403/2789] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:49.696257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ab4e7-4d85-4132-aa6c-7b4a271dc3e5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.696429 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-751ab4e7-4d85-4132-aa6c-7b4a271dc3e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.699510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e62e6-3ee8-4f52-906b-bb1274c38b75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.700978 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657e62e6-3ee8-4f52-906b-bb1274c38b75 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.709676 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751ab4e7-4d85-4132-aa6c-7b4a271dc3e5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.710631 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1387/2790] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:49.711585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e62e6-3ee8-4f52-906b-bb1274c38b75 service cinder] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.715500 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.716228 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.716551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.716987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] PATH_INFO: `/v3/projects/060729b788de40cc8d4d5b8d14db3e23` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.718105 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=060729b788de40cc8d4d5b8d14db3e23)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.723565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657e62e6-3ee8-4f52-906b-bb1274c38b75 service cinder] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.732477 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.734065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211902c-42bc-4e48-b955-ab311fe210ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.735147 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211902c-42bc-4e48-b955-ab311fe210ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.735425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211902c-42bc-4e48-b955-ab311fe210ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.735862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5211902c-42bc-4e48-b955-ab311fe210ca admin admin] PATH_INFO: `/v3/projects/59ca478c57324e6296b2341f05ff4b2e/users/b5f6a70d67e0441c99d312ea8acc25f2/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.753570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211902c-42bc-4e48-b955-ab311fe210ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ca478c57324e6296b2341f05ff4b2e, user_id=b5f6a70d67e0441c99d312ea8acc25f2, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.756351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5211902c-42bc-4e48-b955-ab311fe210ca admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.763463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:49.772298 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1388/2791] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:49 2022] PUT /identity/v3/projects/59ca478c57324e6296b2341f05ff4b2e/users/b5f6a70d67e0441c99d312ea8acc25f2/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:49.776396 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211902c-42bc-4e48-b955-ab311fe210ca admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.791435 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '060729b788de40cc8d4d5b8d14db3e23', 'request_id': 'req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.797760 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '060729b788de40cc8d4d5b8d14db3e23', 'request_id': 'req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.809375 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '060729b788de40cc8d4d5b8d14db3e23', 'request_id': 'req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.823395 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] The token cache is being invalidate because project 060729b788de40cc8d4d5b8d14db3e23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81589) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:25:49.823738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.824558 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-27a7e73c-6bef-4fb8-9c58-4e9fd3614ea8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:25:49.825923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1404/2792] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:25:49 2022] DELETE /identity/v3/projects/060729b788de40cc8d4d5b8d14db3e23 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:49.830548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8699d-f8fc-467c-86ca-241b3cd9a73c None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.830899 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8699d-f8fc-467c-86ca-241b3cd9a73c None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.831244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8699d-f8fc-467c-86ca-241b3cd9a73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.847107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5211902c-42bc-4e48-b955-ab311fe210ca admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.852588 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b6f8699d-f8fc-467c-86ca-241b3cd9a73c None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:49.853879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b6f8699d-f8fc-467c-86ca-241b3cd9a73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.910290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808901f-d2e5-4f0e-9468-21d811c48b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.910904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808901f-d2e5-4f0e-9468-21d811c48b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.911168 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808901f-d2e5-4f0e-9468-21d811c48b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.911466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a808901f-d2e5-4f0e-9468-21d811c48b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.912268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808901f-d2e5-4f0e-9468-21d811c48b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.914534 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a808901f-d2e5-4f0e-9468-21d811c48b70 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.925144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1389/2793] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:25:49.933827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1405/2794] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:25:49.934025 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808901f-d2e5-4f0e-9468-21d811c48b70 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.935374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fa6f4-f433-47d2-956e-037263e88f9b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.935630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a808901f-d2e5-4f0e-9468-21d811c48b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.935752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fa6f4-f433-47d2-956e-037263e88f9b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.935955 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fa6f4-f433-47d2-956e-037263e88f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.949017 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a808901f-d2e5-4f0e-9468-21d811c48b70 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.954446 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb540d-b090-405c-aeb4-ebe0c8a2d6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:49.955243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb540d-b090-405c-aeb4-ebe0c8a2d6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.955681 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb540d-b090-405c-aeb4-ebe0c8a2d6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.956126 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb540d-b090-405c-aeb4-ebe0c8a2d6e9 admin admin] PATH_INFO: `/v3/projects/59ca478c57324e6296b2341f05ff4b2e/users/b5f6a70d67e0441c99d312ea8acc25f2/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:49.956862 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-984fa6f4-f433-47d2-956e-037263e88f9b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:49.958157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-984fa6f4-f433-47d2-956e-037263e88f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:49.959603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb540d-b090-405c-aeb4-ebe0c8a2d6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ca478c57324e6296b2341f05ff4b2e, user_id=b5f6a70d67e0441c99d312ea8acc25f2, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:49.963260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb540d-b090-405c-aeb4-ebe0c8a2d6e9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:49.967378 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1406/2795] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:25:49.972015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fa6f4-f433-47d2-956e-037263e88f9b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:49.977474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1390/2796] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:49 2022] PUT /identity/v3/projects/59ca478c57324e6296b2341f05ff4b2e/users/b5f6a70d67e0441c99d312ea8acc25f2/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:25:49.983192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fa6f4-f433-47d2-956e-037263e88f9b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:49.989649 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb4e8a-fadf-47b5-adfa-ef009a2d5171 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:49.989931 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb4e8a-fadf-47b5-adfa-ef009a2d5171 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:49.990202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb4e8a-fadf-47b5-adfa-ef009a2d5171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:50.023197 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-32fb4e8a-fadf-47b5-adfa-ef009a2d5171 None None] MFA Rules not processed for user `b5f6a70d67e0441c99d312ea8acc25f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:50.024441 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-32fb4e8a-fadf-47b5-adfa-ef009a2d5171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:50.052047 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ee176-c2e0-4b3b-a70e-0c38da9fdb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:50.052670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ee176-c2e0-4b3b-a70e-0c38da9fdb7c admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:50.052943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ee176-c2e0-4b3b-a70e-0c38da9fdb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:50.053583 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ee176-c2e0-4b3b-a70e-0c38da9fdb7c admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:50.054675 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ee176-c2e0-4b3b-a70e-0c38da9fdb7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:25:50.055032 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ee176-c2e0-4b3b-a70e-0c38da9fdb7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:50.056776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ee176-c2e0-4b3b-a70e-0c38da9fdb7c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:50.070527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1407/2797] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:25:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 10 14:25:50.071567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:50.074898 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ee176-c2e0-4b3b-a70e-0c38da9fdb7c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:50.084799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ee176-c2e0-4b3b-a70e-0c38da9fdb7c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:50.090107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d4745-4022-43df-b314-fc9bf0c8489c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:50.090701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d4745-4022-43df-b314-fc9bf0c8489c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:50.090976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d4745-4022-43df-b314-fc9bf0c8489c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:50.091361 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264d4745-4022-43df-b314-fc9bf0c8489c admin admin] PATH_INFO: `/v3/projects` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:50.092227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d4745-4022-43df-b314-fc9bf0c8489c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:50.094426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264d4745-4022-43df-b314-fc9bf0c8489c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:50.111288 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1391/2798] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:49 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:50.124920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb4e8a-fadf-47b5-adfa-ef009a2d5171 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:50.127239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-32fb4e8a-fadf-47b5-adfa-ef009a2d5171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:50.134092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1408/2799] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:25:50 2022] POST /identity/v3/projects => generated 369 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:25:50.138434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d4745-4022-43df-b314-fc9bf0c8489c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:50.161082 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:50.203076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb4e8a-fadf-47b5-adfa-ef009a2d5171 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:50.220736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264d4745-4022-43df-b314-fc9bf0c8489c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:50.226193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb93b8-d0b0-44ef-9e39-3ab63dacf90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:50.226790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb93b8-d0b0-44ef-9e39-3ab63dacf90e admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:50.227113 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb93b8-d0b0-44ef-9e39-3ab63dacf90e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:50.227468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb93b8-d0b0-44ef-9e39-3ab63dacf90e admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:50.228130 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb93b8-d0b0-44ef-9e39-3ab63dacf90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:50.230300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb93b8-d0b0-44ef-9e39-3ab63dacf90e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:50.252407 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1409/2800] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:25:50 2022] POST /identity/v3/users => generated 285 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:25:50.256508 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb93b8-d0b0-44ef-9e39-3ab63dacf90e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:50.259245 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429319c-2f06-434f-b96f-669b8eaf89fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:50.259884 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429319c-2f06-434f-b96f-669b8eaf89fa service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:50.260161 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429319c-2f06-434f-b96f-669b8eaf89fa service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:50.260433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429319c-2f06-434f-b96f-669b8eaf89fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:50.262382 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a429319c-2f06-434f-b96f-669b8eaf89fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:50.266874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb93b8-d0b0-44ef-9e39-3ab63dacf90e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:50.272159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ec3e5-f84b-4dc1-b7cd-64e372c9a9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:50.273367 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ec3e5-f84b-4dc1-b7cd-64e372c9a9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:50.273655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ec3e5-f84b-4dc1-b7cd-64e372c9a9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:50.273933 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ec3e5-f84b-4dc1-b7cd-64e372c9a9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:50.274573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ec3e5-f84b-4dc1-b7cd-64e372c9a9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:50.276289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5ec3e5-f84b-4dc1-b7cd-64e372c9a9f0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:50.287346 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1410/2801] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:25:50.291282 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ec3e5-f84b-4dc1-b7cd-64e372c9a9f0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:50.300823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ec3e5-f84b-4dc1-b7cd-64e372c9a9f0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:50.305601 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c20ed-783f-47dd-8f03-22407be123e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:50.306181 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c20ed-783f-47dd-8f03-22407be123e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:50.306462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c20ed-783f-47dd-8f03-22407be123e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:50.306736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c20ed-783f-47dd-8f03-22407be123e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:50.307369 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c20ed-783f-47dd-8f03-22407be123e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:50.308996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c20ed-783f-47dd-8f03-22407be123e7 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:50.322473 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1411/2802] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:25:50.324092 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c20ed-783f-47dd-8f03-22407be123e7 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:50.326432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429319c-2f06-434f-b96f-669b8eaf89fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:50.328032 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429319c-2f06-434f-b96f-669b8eaf89fa service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:50.333571 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c20ed-783f-47dd-8f03-22407be123e7 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:50.337341 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1392/2803] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:50.338287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2991c-2bb2-4ce3-a6fd-202dc3d26ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:50.338981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2991c-2bb2-4ce3-a6fd-202dc3d26ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:50.339259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2991c-2bb2-4ce3-a6fd-202dc3d26ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:50.339557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b2991c-2bb2-4ce3-a6fd-202dc3d26ab5 admin admin] PATH_INFO: `/v3/projects/f2a216efa4a64d87bd5136b8e8071f37/users/b3b88152b4e842b58bda0a2b5c9c886f/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:50.356162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2991c-2bb2-4ce3-a6fd-202dc3d26ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2a216efa4a64d87bd5136b8e8071f37, user_id=b3b88152b4e842b58bda0a2b5c9c886f, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:50.358821 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b2991c-2bb2-4ce3-a6fd-202dc3d26ab5 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:50.373075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1412/2804] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:25:50 2022] PUT /identity/v3/projects/f2a216efa4a64d87bd5136b8e8071f37/users/b3b88152b4e842b58bda0a2b5c9c886f/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:25:50.385797 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af280c20-0378-48b5-b414-ca82b4dc8abc None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:50.386083 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af280c20-0378-48b5-b414-ca82b4dc8abc None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:50.386360 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af280c20-0378-48b5-b414-ca82b4dc8abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:50.415251 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-af280c20-0378-48b5-b414-ca82b4dc8abc None None] MFA Rules not processed for user `b3b88152b4e842b58bda0a2b5c9c886f`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:25:50.416539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-af280c20-0378-48b5-b414-ca82b4dc8abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:50.451886 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:25:50.485668 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1393/2805] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:25:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:25:50.499552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2991c-2bb2-4ce3-a6fd-202dc3d26ab5 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:50.501448 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-07b2991c-2bb2-4ce3-a6fd-202dc3d26ab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:25:50.598328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b2991c-2bb2-4ce3-a6fd-202dc3d26ab5 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:50.598328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba6f99-44ea-4e68-819b-836ca4ae768b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cf75b583add44b6b87bff5dc67080d1c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:50.598328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba6f99-44ea-4e68-819b-836ca4ae768b service glance] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:50.598328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba6f99-44ea-4e68-819b-836ca4ae768b service glance] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:50.598766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba6f99-44ea-4e68-819b-836ca4ae768b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:50.605635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba6f99-44ea-4e68-819b-836ca4ae768b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:50.607124 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba6f99-44ea-4e68-819b-836ca4ae768b service glance] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:50.617209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1413/2806] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:25:57.047363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af280c20-0378-48b5-b414-ca82b4dc8abc None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:25:57.106381 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af280c20-0378-48b5-b414-ca82b4dc8abc None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:25:57.110058 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb9c0e-e1de-42c1-955f-6ccef24fe292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:25:57.110673 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb9c0e-e1de-42c1-955f-6ccef24fe292 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:25:57.110969 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb9c0e-e1de-42c1-955f-6ccef24fe292 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:25:57.111244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb9c0e-e1de-42c1-955f-6ccef24fe292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:25:57.183474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb9c0e-e1de-42c1-955f-6ccef24fe292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:25:57.185053 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb9c0e-e1de-42c1-955f-6ccef24fe292 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:25:57.194857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1394/2807] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:25:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:00.612377 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=77,keystone:INSERT=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:07.190452 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=61 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:13.615904 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba6f99-44ea-4e68-819b-836ca4ae768b service glance] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:13.617528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-9dba6f99-44ea-4e68-819b-836ca4ae768b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:13.689267 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba6f99-44ea-4e68-819b-836ca4ae768b service glance] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:13.693936 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:13.694567 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:13.694840 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:13.695109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] PATH_INFO: `/v3/users/c73a4aa58ec848258c4b47d9aad51a0b` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:13.696287 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c73a4aa58ec848258c4b47d9aad51a0b)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:13.698817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:13.747579 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c73a4aa58ec848258c4b47d9aad51a0b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:13.756274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c73a4aa58ec848258c4b47d9aad51a0b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:13.756912 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c73a4aa58ec848258c4b47d9aad51a0b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:13.757545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c73a4aa58ec848258c4b47d9aad51a0b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:13.765400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c73a4aa58ec848258c4b47d9aad51a0b'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:13.785609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1414/2808] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:26:13 2022] DELETE /identity/v3/users/c73a4aa58ec848258c4b47d9aad51a0b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:26:13.790193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7003fa4-595c-4fee-8bfa-893f199d9a4b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:13.790793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7003fa4-595c-4fee-8bfa-893f199d9a4b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:13.790915 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7003fa4-595c-4fee-8bfa-893f199d9a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:13.810639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-a7003fa4-595c-4fee-8bfa-893f199d9a4b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:13.812105 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a7003fa4-595c-4fee-8bfa-893f199d9a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:13.876745 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1395/2809] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:26:13.887754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:13.889154 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:13.946630 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfdbdb6-681c-47a2-8e22-69a6e3cb5b15 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:13.950732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f378e04-4a44-4c14-968e-ef01db60e17d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:13.951322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378e04-4a44-4c14-968e-ef01db60e17d service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:13.951637 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378e04-4a44-4c14-968e-ef01db60e17d service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:13.951906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f378e04-4a44-4c14-968e-ef01db60e17d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:13.960806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f378e04-4a44-4c14-968e-ef01db60e17d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:13.962224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f378e04-4a44-4c14-968e-ef01db60e17d service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:13.971834 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1415/2810] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:14.236943 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7003fa4-595c-4fee-8bfa-893f199d9a4b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:14.239497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a7003fa4-595c-4fee-8bfa-893f199d9a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:14.256152 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7003fa4-595c-4fee-8bfa-893f199d9a4b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:14.261238 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:14.261980 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:14.262150 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:14.262399 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] PATH_INFO: `/v3/projects/b9d256f4805f4d45af9a4df1d37f4246` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:14.263527 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d256f4805f4d45af9a4df1d37f4246)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:14.265439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:14.318425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d256f4805f4d45af9a4df1d37f4246', 'request_id': 'req-15c1e30f-facf-4a1a-be7e-93228da2aaa9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:14.325469 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d256f4805f4d45af9a4df1d37f4246', 'request_id': 'req-15c1e30f-facf-4a1a-be7e-93228da2aaa9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:14.330892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d256f4805f4d45af9a4df1d37f4246', 'request_id': 'req-15c1e30f-facf-4a1a-be7e-93228da2aaa9'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:14.342094 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] The token cache is being invalidate because project b9d256f4805f4d45af9a4df1d37f4246 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:26:14.342378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:14.342996 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-15c1e30f-facf-4a1a-be7e-93228da2aaa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:14.344492 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1396/2811] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:26:14 2022] DELETE /identity/v3/projects/b9d256f4805f4d45af9a4df1d37f4246 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:15.930923 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f378e04-4a44-4c14-968e-ef01db60e17d service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:15.935171 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1f378e04-4a44-4c14-968e-ef01db60e17d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:15.964859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:16.056649 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f378e04-4a44-4c14-968e-ef01db60e17d service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:16.132489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a00094-8294-446b-875a-7a66bff021a1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:16.133233 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a00094-8294-446b-875a-7a66bff021a1 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:16.133660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a00094-8294-446b-875a-7a66bff021a1 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:16.134031 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a00094-8294-446b-875a-7a66bff021a1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:16.136741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d1a00094-8294-446b-875a-7a66bff021a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:16.149391 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a00094-8294-446b-875a-7a66bff021a1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:16.151229 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a00094-8294-446b-875a-7a66bff021a1 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:16.162705 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1416/2812] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:22.704674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b8f72-6b92-4d02-a2cb-bb4e86c6dfee None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:22.705123 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b8f72-6b92-4d02-a2cb-bb4e86c6dfee None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:22.705231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b8f72-6b92-4d02-a2cb-bb4e86c6dfee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:22.724131 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-aa1b8f72-6b92-4d02-a2cb-bb4e86c6dfee None None] MFA Rules not processed for user `f4638a1936f5467db502bda82320b113`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:22.725312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-aa1b8f72-6b92-4d02-a2cb-bb4e86c6dfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:22.782545 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1397/2813] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:23.097255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a00094-8294-446b-875a-7a66bff021a1 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:23.098670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d1a00094-8294-446b-875a-7a66bff021a1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:23.155156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a00094-8294-446b-875a-7a66bff021a1 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:23.158668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca3efd-3e16-4b83-9a03-5e03159fd21c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:23.159255 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca3efd-3e16-4b83-9a03-5e03159fd21c service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:23.159553 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca3efd-3e16-4b83-9a03-5e03159fd21c service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:23.159825 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ca3efd-3e16-4b83-9a03-5e03159fd21c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:23.161490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-46ca3efd-3e16-4b83-9a03-5e03159fd21c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:23.171009 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca3efd-3e16-4b83-9a03-5e03159fd21c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:23.172463 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ca3efd-3e16-4b83-9a03-5e03159fd21c service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:23.180320 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1417/2814] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:23.671339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b8f72-6b92-4d02-a2cb-bb4e86c6dfee None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:23.672940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-aa1b8f72-6b92-4d02-a2cb-bb4e86c6dfee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:23.710171 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:23.753558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b8f72-6b92-4d02-a2cb-bb4e86c6dfee None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:23.758456 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f47d2-c3d5-4638-b1b0-09a838365901 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:23.759816 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f47d2-c3d5-4638-b1b0-09a838365901 service placement] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:23.760092 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f47d2-c3d5-4638-b1b0-09a838365901 service placement] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:23.760365 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f47d2-c3d5-4638-b1b0-09a838365901 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:23.762128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-de2f47d2-c3d5-4638-b1b0-09a838365901 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:23.775175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f47d2-c3d5-4638-b1b0-09a838365901 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:23.776824 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f47d2-c3d5-4638-b1b0-09a838365901 service placement] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:23.787110 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1398/2815] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:33.176717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=78 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:33.472879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca3efd-3e16-4b83-9a03-5e03159fd21c service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:33.474455 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-46ca3efd-3e16-4b83-9a03-5e03159fd21c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:33.540283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ca3efd-3e16-4b83-9a03-5e03159fd21c service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:33.545449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:33.546064 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:33.546334 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:33.546600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] PATH_INFO: `/v3/users/f080ef0e6d0b46b5876316042e99d910` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:33.547738 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f080ef0e6d0b46b5876316042e99d910)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:33.549653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:33.594382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f080ef0e6d0b46b5876316042e99d910'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:33.600967 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f080ef0e6d0b46b5876316042e99d910'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:33.601841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f080ef0e6d0b46b5876316042e99d910'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:33.602511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f080ef0e6d0b46b5876316042e99d910'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:33.610101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f080ef0e6d0b46b5876316042e99d910'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:33.628446 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1418/2816] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:26:33 2022] DELETE /identity/v3/users/f080ef0e6d0b46b5876316042e99d910 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:33.632914 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49214d2-695f-49d8-b5ce-0f412738e064 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:33.633402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49214d2-695f-49d8-b5ce-0f412738e064 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:33.633526 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49214d2-695f-49d8-b5ce-0f412738e064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:33.652323 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-e49214d2-695f-49d8-b5ce-0f412738e064 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:33.653500 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e49214d2-695f-49d8-b5ce-0f412738e064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:33.711523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1399/2817] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:33.721158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:33.722564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:33.780140 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0ce9a-4be3-4ac8-ac45-d1fef802e4d8 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:33.783811 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5243a-e3af-440d-9495-d1133f948b93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:33.784413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5243a-e3af-440d-9495-d1133f948b93 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:33.784683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5243a-e3af-440d-9495-d1133f948b93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:33.784957 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5243a-e3af-440d-9495-d1133f948b93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:33.794618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5243a-e3af-440d-9495-d1133f948b93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:33.796102 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5243a-e3af-440d-9495-d1133f948b93 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:33.805452 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1419/2818] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:34.136263 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49214d2-695f-49d8-b5ce-0f412738e064 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:34.137665 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-e49214d2-695f-49d8-b5ce-0f412738e064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:34.149679 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49214d2-695f-49d8-b5ce-0f412738e064 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:34.155226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:34.155971 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:34.156447 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:34.156783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] PATH_INFO: `/v3/projects/4cdf4f2f64fc4b91b144e1ea031115a1` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:34.157972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cdf4f2f64fc4b91b144e1ea031115a1)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:34.160223 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:34.208974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cdf4f2f64fc4b91b144e1ea031115a1', 'request_id': 'req-a9558dab-3fbd-4726-81c2-fbf89dfd460d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.216899 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cdf4f2f64fc4b91b144e1ea031115a1', 'request_id': 'req-a9558dab-3fbd-4726-81c2-fbf89dfd460d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.222424 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cdf4f2f64fc4b91b144e1ea031115a1', 'request_id': 'req-a9558dab-3fbd-4726-81c2-fbf89dfd460d'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.233371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] The token cache is being invalidate because project 4cdf4f2f64fc4b91b144e1ea031115a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:26:34.233662 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.234318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.235828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1400/2819] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:26:34 2022] DELETE /identity/v3/projects/4cdf4f2f64fc4b91b144e1ea031115a1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:26:34.240050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5243a-e3af-440d-9495-d1133f948b93 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:34.249692 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5243a-e3af-440d-9495-d1133f948b93 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:34.268053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:34.307613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:34.308227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:34.308501 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:34.308769 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] PATH_INFO: `/v3/users/b5f6a70d67e0441c99d312ea8acc25f2` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:34.309871 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5f6a70d67e0441c99d312ea8acc25f2)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:34.311849 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:34.357513 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5f6a70d67e0441c99d312ea8acc25f2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.364634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5f6a70d67e0441c99d312ea8acc25f2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.365238 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5f6a70d67e0441c99d312ea8acc25f2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.365839 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5f6a70d67e0441c99d312ea8acc25f2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.372651 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-9c556fa5-3595-4802-9690-0fef363ba82e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5f6a70d67e0441c99d312ea8acc25f2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.389609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1420/2820] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:26:34 2022] DELETE /identity/v3/users/b5f6a70d67e0441c99d312ea8acc25f2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:34.660869 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:34.662425 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:34.677349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9558dab-3fbd-4726-81c2-fbf89dfd460d admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:34.720461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:34.741901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:34.742466 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:34.742746 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:34.743019 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] PATH_INFO: `/v3/projects/59ca478c57324e6296b2341f05ff4b2e` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:34.744680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ca478c57324e6296b2341f05ff4b2e)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:34.746561 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:34.792818 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ca478c57324e6296b2341f05ff4b2e', 'request_id': 'req-b6b3779f-0829-4976-828c-5ba1b2be4827'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.800519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ca478c57324e6296b2341f05ff4b2e', 'request_id': 'req-b6b3779f-0829-4976-828c-5ba1b2be4827'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.806949 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ca478c57324e6296b2341f05ff4b2e', 'request_id': 'req-b6b3779f-0829-4976-828c-5ba1b2be4827'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.818146 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] The token cache is being invalidate because project 59ca478c57324e6296b2341f05ff4b2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:26:34.818432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.819061 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-b6b3779f-0829-4976-828c-5ba1b2be4827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:34.820592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1401/2821] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:26:34 2022] DELETE /identity/v3/projects/59ca478c57324e6296b2341f05ff4b2e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:34.826190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94527f-b468-48e5-b540-125a4a09d237 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:34.826521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94527f-b468-48e5-b540-125a4a09d237 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:34.826793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c94527f-b468-48e5-b540-125a4a09d237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:34.968701 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8c94527f-b468-48e5-b540-125a4a09d237 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:34.969935 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8c94527f-b468-48e5-b540-125a4a09d237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:35.030216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1421/2822] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:35.037929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49a91-c61a-48cd-b40e-1160776d609f None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.038240 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49a91-c61a-48cd-b40e-1160776d609f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.038537 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef49a91-c61a-48cd-b40e-1160776d609f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.059184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-fef49a91-c61a-48cd-b40e-1160776d609f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:35.060418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-fef49a91-c61a-48cd-b40e-1160776d609f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:35.068145 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1402/2823] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:35.072854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94527f-b468-48e5-b540-125a4a09d237 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.082527 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c94527f-b468-48e5-b540-125a4a09d237 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.087120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57b7c3-45e5-4e45-95f5-7ecdb8522a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.088458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57b7c3-45e5-4e45-95f5-7ecdb8522a5e admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.088782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57b7c3-45e5-4e45-95f5-7ecdb8522a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.089069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57b7c3-45e5-4e45-95f5-7ecdb8522a5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.089698 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57b7c3-45e5-4e45-95f5-7ecdb8522a5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:26:35.089992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57b7c3-45e5-4e45-95f5-7ecdb8522a5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.091596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57b7c3-45e5-4e45-95f5-7ecdb8522a5e admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.106192 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1422/2824] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:26:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:26:35.107489 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef49a91-c61a-48cd-b40e-1160776d609f None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.116849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef49a91-c61a-48cd-b40e-1160776d609f None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.121647 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4338b20-003a-4fd5-90dc-1ddbc182cb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.122280 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4338b20-003a-4fd5-90dc-1ddbc182cb63 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.122556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4338b20-003a-4fd5-90dc-1ddbc182cb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.122821 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4338b20-003a-4fd5-90dc-1ddbc182cb63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.123487 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4338b20-003a-4fd5-90dc-1ddbc182cb63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.125411 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4338b20-003a-4fd5-90dc-1ddbc182cb63 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.158868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1403/2825] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:26:35 2022] POST /identity/v3/projects => generated 369 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:26:35.160100 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57b7c3-45e5-4e45-95f5-7ecdb8522a5e admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.170111 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57b7c3-45e5-4e45-95f5-7ecdb8522a5e admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.196253 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:35.232595 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28949ccb-7ddd-4610-9df0-69856ba5a03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.233209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28949ccb-7ddd-4610-9df0-69856ba5a03c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.233488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28949ccb-7ddd-4610-9df0-69856ba5a03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.233752 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28949ccb-7ddd-4610-9df0-69856ba5a03c admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.234376 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28949ccb-7ddd-4610-9df0-69856ba5a03c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.236359 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28949ccb-7ddd-4610-9df0-69856ba5a03c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.258434 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1423/2826] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:26:35 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:35.263050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4338b20-003a-4fd5-90dc-1ddbc182cb63 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.273714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4338b20-003a-4fd5-90dc-1ddbc182cb63 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.279206 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376074db-9f3a-4ac8-b338-ee66dfba3809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.279970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376074db-9f3a-4ac8-b338-ee66dfba3809 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.280141 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376074db-9f3a-4ac8-b338-ee66dfba3809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.280428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376074db-9f3a-4ac8-b338-ee66dfba3809 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.280999 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376074db-9f3a-4ac8-b338-ee66dfba3809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.282600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376074db-9f3a-4ac8-b338-ee66dfba3809 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.298697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1404/2827] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:35.302602 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28949ccb-7ddd-4610-9df0-69856ba5a03c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.313358 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28949ccb-7ddd-4610-9df0-69856ba5a03c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.318086 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9d125-8316-4f28-ba6a-65c798c596cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.318696 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d125-8316-4f28-ba6a-65c798c596cb admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.318974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d125-8316-4f28-ba6a-65c798c596cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.319241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d125-8316-4f28-ba6a-65c798c596cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.319985 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9d125-8316-4f28-ba6a-65c798c596cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.321612 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b9d125-8316-4f28-ba6a-65c798c596cb admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.332036 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1424/2828] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:35.336142 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376074db-9f3a-4ac8-b338-ee66dfba3809 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.346004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376074db-9f3a-4ac8-b338-ee66dfba3809 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.350567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d26168-0b03-4cb4-97ff-72caef38a045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.351202 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d26168-0b03-4cb4-97ff-72caef38a045 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.351510 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d26168-0b03-4cb4-97ff-72caef38a045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.351784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d26168-0b03-4cb4-97ff-72caef38a045 admin admin] PATH_INFO: `/v3/projects/146e14ac87ba45b1b7ea5579d1a97e0c/users/b3c84d5ebf4447499dec5266b74235d2/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.366436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d26168-0b03-4cb4-97ff-72caef38a045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=146e14ac87ba45b1b7ea5579d1a97e0c, user_id=b3c84d5ebf4447499dec5266b74235d2, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.369844 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d26168-0b03-4cb4-97ff-72caef38a045 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.382182 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1405/2829] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:26:35 2022] PUT /identity/v3/projects/146e14ac87ba45b1b7ea5579d1a97e0c/users/b3c84d5ebf4447499dec5266b74235d2/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:35.394269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2553c-c399-4a2c-ad5d-eb197c27223e None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.394548 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2553c-c399-4a2c-ad5d-eb197c27223e None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.394818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd2553c-c399-4a2c-ad5d-eb197c27223e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.423512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-0cd2553c-c399-4a2c-ad5d-eb197c27223e None None] MFA Rules not processed for user `b3c84d5ebf4447499dec5266b74235d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:35.424717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-0cd2553c-c399-4a2c-ad5d-eb197c27223e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:35.490506 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1425/2830] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:35.495035 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d26168-0b03-4cb4-97ff-72caef38a045 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.555916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d26168-0b03-4cb4-97ff-72caef38a045 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.560600 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02d85e-3a41-40e0-b58a-c3466eba47ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.561190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02d85e-3a41-40e0-b58a-c3466eba47ad admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.561463 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02d85e-3a41-40e0-b58a-c3466eba47ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.561729 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02d85e-3a41-40e0-b58a-c3466eba47ad admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.562349 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02d85e-3a41-40e0-b58a-c3466eba47ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.564269 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e02d85e-3a41-40e0-b58a-c3466eba47ad admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.579173 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:35.593714 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1406/2831] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:26:35 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:26:35.597757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2553c-c399-4a2c-ad5d-eb197c27223e None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.608874 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd2553c-c399-4a2c-ad5d-eb197c27223e None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.663756 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d56fe7-110b-4545-aa16-19f6a9449b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.664356 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d56fe7-110b-4545-aa16-19f6a9449b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.664633 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d56fe7-110b-4545-aa16-19f6a9449b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.664901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d56fe7-110b-4545-aa16-19f6a9449b4c admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.665584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d56fe7-110b-4545-aa16-19f6a9449b4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.668343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d56fe7-110b-4545-aa16-19f6a9449b4c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.689730 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1426/2832] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:26:35 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:35.693567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02d85e-3a41-40e0-b58a-c3466eba47ad admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.704443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e02d85e-3a41-40e0-b58a-c3466eba47ad admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.709037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb3702-0efb-4f15-9bae-7f3e36b68614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.709618 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb3702-0efb-4f15-9bae-7f3e36b68614 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.709896 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb3702-0efb-4f15-9bae-7f3e36b68614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.710196 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb3702-0efb-4f15-9bae-7f3e36b68614 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.710826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb3702-0efb-4f15-9bae-7f3e36b68614 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.712458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb3702-0efb-4f15-9bae-7f3e36b68614 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.723727 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1407/2833] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:35.728070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d56fe7-110b-4545-aa16-19f6a9449b4c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.738273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d56fe7-110b-4545-aa16-19f6a9449b4c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.742845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe94b1-4e27-4e29-b40a-b4463983e4e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.743507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe94b1-4e27-4e29-b40a-b4463983e4e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.743783 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe94b1-4e27-4e29-b40a-b4463983e4e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.744055 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe94b1-4e27-4e29-b40a-b4463983e4e1 admin admin] PATH_INFO: `/v3/projects/839241a5b39947b4a31e647af4b8f92a/users/9bb3b35ffdfb41d5bb71cf0c5450f811/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.758520 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe94b1-4e27-4e29-b40a-b4463983e4e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=839241a5b39947b4a31e647af4b8f92a, user_id=9bb3b35ffdfb41d5bb71cf0c5450f811, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.761807 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe94b1-4e27-4e29-b40a-b4463983e4e1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.775632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1427/2834] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:26:35 2022] PUT /identity/v3/projects/839241a5b39947b4a31e647af4b8f92a/users/9bb3b35ffdfb41d5bb71cf0c5450f811/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:35.780178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb3702-0efb-4f15-9bae-7f3e36b68614 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.789945 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb3702-0efb-4f15-9bae-7f3e36b68614 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.846135 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685fef93-5183-407e-9d85-316d94dd2736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.846781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fef93-5183-407e-9d85-316d94dd2736 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.847079 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fef93-5183-407e-9d85-316d94dd2736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.847350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685fef93-5183-407e-9d85-316d94dd2736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.848015 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fef93-5183-407e-9d85-316d94dd2736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.849757 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685fef93-5183-407e-9d85-316d94dd2736 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.860894 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1408/2835] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:35.865329 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe94b1-4e27-4e29-b40a-b4463983e4e1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:35.871521 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:35.877833 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe94b1-4e27-4e29-b40a-b4463983e4e1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:35.882799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa04fa-0217-4e26-8cd5-b6e02174f5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:35.883437 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa04fa-0217-4e26-8cd5-b6e02174f5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.883734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa04fa-0217-4e26-8cd5-b6e02174f5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.884003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa04fa-0217-4e26-8cd5-b6e02174f5f1 admin admin] PATH_INFO: `/v3/projects/839241a5b39947b4a31e647af4b8f92a/users/9bb3b35ffdfb41d5bb71cf0c5450f811/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.886217 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa04fa-0217-4e26-8cd5-b6e02174f5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=839241a5b39947b4a31e647af4b8f92a, user_id=9bb3b35ffdfb41d5bb71cf0c5450f811, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:35.888946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa04fa-0217-4e26-8cd5-b6e02174f5f1 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:35.899304 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1428/2836] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:26:35 2022] PUT /identity/v3/projects/839241a5b39947b4a31e647af4b8f92a/users/9bb3b35ffdfb41d5bb71cf0c5450f811/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:35.909497 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae991a71-4ff1-41a9-bcdd-9786aba1ea13 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:35.909778 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae991a71-4ff1-41a9-bcdd-9786aba1ea13 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:35.910049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae991a71-4ff1-41a9-bcdd-9786aba1ea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:35.939272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-ae991a71-4ff1-41a9-bcdd-9786aba1ea13 None None] MFA Rules not processed for user `9bb3b35ffdfb41d5bb71cf0c5450f811`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:35.940641 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ae991a71-4ff1-41a9-bcdd-9786aba1ea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:36.009592 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1409/2837] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:36.021328 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa04fa-0217-4e26-8cd5-b6e02174f5f1 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:36.023051 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-18aa04fa-0217-4e26-8cd5-b6e02174f5f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:36.080158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa04fa-0217-4e26-8cd5-b6e02174f5f1 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:36.083703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f479886-49b4-43df-8967-75de0ba5d887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:36.084289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f479886-49b4-43df-8967-75de0ba5d887 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:36.084561 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f479886-49b4-43df-8967-75de0ba5d887 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:36.084826 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f479886-49b4-43df-8967-75de0ba5d887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:36.095096 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f479886-49b4-43df-8967-75de0ba5d887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:36.096615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f479886-49b4-43df-8967-75de0ba5d887 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:36.106586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1429/2838] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:36 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:36.156186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae991a71-4ff1-41a9-bcdd-9786aba1ea13 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:36.165878 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae991a71-4ff1-41a9-bcdd-9786aba1ea13 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:36.170406 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7b241e-c213-48b4-9291-8de5fedb6294 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:36.171000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b241e-c213-48b4-9291-8de5fedb6294 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:36.171275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b241e-c213-48b4-9291-8de5fedb6294 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:36.171576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7b241e-c213-48b4-9291-8de5fedb6294 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:36.226078 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b241e-c213-48b4-9291-8de5fedb6294 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:36.227519 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7b241e-c213-48b4-9291-8de5fedb6294 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:36.237107 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1410/2839] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:39.560594 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f479886-49b4-43df-8967-75de0ba5d887 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:39.622624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f479886-49b4-43df-8967-75de0ba5d887 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:39.627089 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:39.627741 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:39.628034 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:39.628284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] PATH_INFO: `/v3/users/b3c84d5ebf4447499dec5266b74235d2` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:39.629374 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3c84d5ebf4447499dec5266b74235d2)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:39.631605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:39.678200 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3c84d5ebf4447499dec5266b74235d2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:39.685352 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3c84d5ebf4447499dec5266b74235d2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:39.686035 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3c84d5ebf4447499dec5266b74235d2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:39.686665 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3c84d5ebf4447499dec5266b74235d2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:39.696261 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3c84d5ebf4447499dec5266b74235d2'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:39.714643 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1430/2840] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:26:39 2022] DELETE /identity/v3/users/b3c84d5ebf4447499dec5266b74235d2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:39.719130 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f350bd-62ae-4d7b-b0d6-10fc0c3beeb8 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:39.719638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f350bd-62ae-4d7b-b0d6-10fc0c3beeb8 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:39.719741 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f350bd-62ae-4d7b-b0d6-10fc0c3beeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:39.726982 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:39.743654 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-26f350bd-62ae-4d7b-b0d6-10fc0c3beeb8 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:39.745206 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-26f350bd-62ae-4d7b-b0d6-10fc0c3beeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:39.804190 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1411/2841] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:39.814446 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:39.815924 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:39.829953 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:39.878070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c2d71-36c9-404a-8ce6-6ea0c9d98e8c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:39.881728 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f36404-5c28-412e-8db5-01457bb6dfdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:39.882331 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f36404-5c28-412e-8db5-01457bb6dfdc service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:39.882608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f36404-5c28-412e-8db5-01457bb6dfdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:39.882879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f36404-5c28-412e-8db5-01457bb6dfdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:39.893128 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f36404-5c28-412e-8db5-01457bb6dfdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:39.894569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f36404-5c28-412e-8db5-01457bb6dfdc service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:39.904702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1431/2842] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:40.176829 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f350bd-62ae-4d7b-b0d6-10fc0c3beeb8 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:40.178289 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-26f350bd-62ae-4d7b-b0d6-10fc0c3beeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:40.190333 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f350bd-62ae-4d7b-b0d6-10fc0c3beeb8 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:40.195229 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:40.195898 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:40.196175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:40.196445 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] PATH_INFO: `/v3/projects/146e14ac87ba45b1b7ea5579d1a97e0c` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:40.198282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=146e14ac87ba45b1b7ea5579d1a97e0c)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:40.200460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:40.248231 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '146e14ac87ba45b1b7ea5579d1a97e0c', 'request_id': 'req-59342e80-ca58-4966-b85d-0bd3ad0337e6'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.255703 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '146e14ac87ba45b1b7ea5579d1a97e0c', 'request_id': 'req-59342e80-ca58-4966-b85d-0bd3ad0337e6'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.262253 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '146e14ac87ba45b1b7ea5579d1a97e0c', 'request_id': 'req-59342e80-ca58-4966-b85d-0bd3ad0337e6'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.273172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] The token cache is being invalidate because project 146e14ac87ba45b1b7ea5579d1a97e0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:26:40.273460 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.274087 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.275747 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1412/2843] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:26:40 2022] DELETE /identity/v3/projects/146e14ac87ba45b1b7ea5579d1a97e0c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:40.280438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f36404-5c28-412e-8db5-01457bb6dfdc service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:40.289837 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f36404-5c28-412e-8db5-01457bb6dfdc service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:40.345074 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:40.345608 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:40.345879 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:40.346147 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] PATH_INFO: `/v3/users/9bb3b35ffdfb41d5bb71cf0c5450f811` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:40.347239 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bb3b35ffdfb41d5bb71cf0c5450f811)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:40.349177 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:40.393303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bb3b35ffdfb41d5bb71cf0c5450f811'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.400382 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bb3b35ffdfb41d5bb71cf0c5450f811'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.401534 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bb3b35ffdfb41d5bb71cf0c5450f811'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.402119 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bb3b35ffdfb41d5bb71cf0c5450f811'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.409050 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-2b4ea61b-7f71-43da-833d-bd9b8a004373 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bb3b35ffdfb41d5bb71cf0c5450f811'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.428319 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1432/2844] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:26:40 2022] DELETE /identity/v3/users/9bb3b35ffdfb41d5bb71cf0c5450f811 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:40.728792 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:40.732048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:40.747958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59342e80-ca58-4966-b85d-0bd3ad0337e6 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:40.811226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:40.811888 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:40.812163 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:40.812431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] PATH_INFO: `/v3/projects/839241a5b39947b4a31e647af4b8f92a` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:40.813535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=839241a5b39947b4a31e647af4b8f92a)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:40.815427 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:40.846226 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:40.862461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '839241a5b39947b4a31e647af4b8f92a', 'request_id': 'req-36ef56f6-323f-4b17-8a10-6e3ca679f066'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.870200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '839241a5b39947b4a31e647af4b8f92a', 'request_id': 'req-36ef56f6-323f-4b17-8a10-6e3ca679f066'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.875589 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '839241a5b39947b4a31e647af4b8f92a', 'request_id': 'req-36ef56f6-323f-4b17-8a10-6e3ca679f066'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.886845 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] The token cache is being invalidate because project 839241a5b39947b4a31e647af4b8f92a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:26:40.887122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.887820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-36ef56f6-323f-4b17-8a10-6e3ca679f066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:40.889286 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1413/2845] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:26:40 2022] DELETE /identity/v3/projects/839241a5b39947b4a31e647af4b8f92a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:40.894225 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852ddd0-38fb-4604-998b-82e719489172 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:40.894695 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852ddd0-38fb-4604-998b-82e719489172 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:40.894812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8852ddd0-38fb-4604-998b-82e719489172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:40.913686 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-8852ddd0-38fb-4604-998b-82e719489172 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:40.914863 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-8852ddd0-38fb-4604-998b-82e719489172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:40.946384 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:40.980668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1433/2846] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:40.990346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e91f6-d227-4edc-b8be-b174fe4330cf None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:40.990648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e91f6-d227-4edc-b8be-b174fe4330cf None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:40.990916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e91f6-d227-4edc-b8be-b174fe4330cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.011904 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-130e91f6-d227-4edc-b8be-b174fe4330cf None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:41.013230 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-130e91f6-d227-4edc-b8be-b174fe4330cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:41.022132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1414/2847] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:26:41.026668 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852ddd0-38fb-4604-998b-82e719489172 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.038336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8852ddd0-38fb-4604-998b-82e719489172 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.044683 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fc120-8176-4c9e-baa8-dcf1f7691d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.045288 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fc120-8176-4c9e-baa8-dcf1f7691d71 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.045616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fc120-8176-4c9e-baa8-dcf1f7691d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.045864 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321fc120-8176-4c9e-baa8-dcf1f7691d71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.046615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fc120-8176-4c9e-baa8-dcf1f7691d71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:26:41.046948 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fc120-8176-4c9e-baa8-dcf1f7691d71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.048655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321fc120-8176-4c9e-baa8-dcf1f7691d71 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.062197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1434/2848] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:26:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:26:41.066529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e91f6-d227-4edc-b8be-b174fe4330cf None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.075916 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e91f6-d227-4edc-b8be-b174fe4330cf None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.082117 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5322142-cb35-461c-88c4-17a699bc69b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.082819 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5322142-cb35-461c-88c4-17a699bc69b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.083167 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5322142-cb35-461c-88c4-17a699bc69b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.083539 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5322142-cb35-461c-88c4-17a699bc69b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.084242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5322142-cb35-461c-88c4-17a699bc69b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.086212 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5322142-cb35-461c-88c4-17a699bc69b8 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.117493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1415/2849] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:26:41 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:41.118660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fc120-8176-4c9e-baa8-dcf1f7691d71 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.128966 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321fc120-8176-4c9e-baa8-dcf1f7691d71 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.194823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ed831-9099-4815-a56c-13508069c4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.194823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ed831-9099-4815-a56c-13508069c4b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.194823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ed831-9099-4815-a56c-13508069c4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.194823 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ed831-9099-4815-a56c-13508069c4b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.195224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ed831-9099-4815-a56c-13508069c4b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.195224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ed831-9099-4815-a56c-13508069c4b3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.211950 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1435/2850] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:26:41 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 10 14:26:41.216686 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5322142-cb35-461c-88c4-17a699bc69b8 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.226967 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5322142-cb35-461c-88c4-17a699bc69b8 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.231803 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a92a4c-7d93-49a5-bf00-a8d123e0b2ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.232400 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a92a4c-7d93-49a5-bf00-a8d123e0b2ae admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.232678 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a92a4c-7d93-49a5-bf00-a8d123e0b2ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.232952 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a92a4c-7d93-49a5-bf00-a8d123e0b2ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.233593 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a92a4c-7d93-49a5-bf00-a8d123e0b2ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.235200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a92a4c-7d93-49a5-bf00-a8d123e0b2ae admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.245482 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1416/2851] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:41.249416 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ed831-9099-4815-a56c-13508069c4b3 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.259088 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ed831-9099-4815-a56c-13508069c4b3 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.264283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a899c98-bec8-4548-80bb-bbf15e6fac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.264861 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a899c98-bec8-4548-80bb-bbf15e6fac78 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.265160 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a899c98-bec8-4548-80bb-bbf15e6fac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.265431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a899c98-bec8-4548-80bb-bbf15e6fac78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.266067 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a899c98-bec8-4548-80bb-bbf15e6fac78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.267682 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a899c98-bec8-4548-80bb-bbf15e6fac78 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.278413 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1436/2852] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:41.282339 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a92a4c-7d93-49a5-bf00-a8d123e0b2ae admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.292307 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a92a4c-7d93-49a5-bf00-a8d123e0b2ae admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.296935 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc630c0d-65e7-4d32-aded-b7368889b61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.297572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc630c0d-65e7-4d32-aded-b7368889b61e admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.297848 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc630c0d-65e7-4d32-aded-b7368889b61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.298121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc630c0d-65e7-4d32-aded-b7368889b61e admin admin] PATH_INFO: `/v3/projects/85faa527e9cd4d19ba25cbafea6fa355/users/10888d2d2b004e18b85e9f28f22b5daf/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.313011 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc630c0d-65e7-4d32-aded-b7368889b61e admin admin] RBAC: Authorizing `identity:create_grant(project_id=85faa527e9cd4d19ba25cbafea6fa355, user_id=10888d2d2b004e18b85e9f28f22b5daf, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.315697 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc630c0d-65e7-4d32-aded-b7368889b61e admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.329375 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1417/2853] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:26:41 2022] PUT /identity/v3/projects/85faa527e9cd4d19ba25cbafea6fa355/users/10888d2d2b004e18b85e9f28f22b5daf/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:41.395300 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2452ef1-877b-4fea-856d-c0987334f793 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.395568 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2452ef1-877b-4fea-856d-c0987334f793 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.395787 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2452ef1-877b-4fea-856d-c0987334f793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.423162 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-b2452ef1-877b-4fea-856d-c0987334f793 None None] MFA Rules not processed for user `10888d2d2b004e18b85e9f28f22b5daf`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:41.424673 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b2452ef1-877b-4fea-856d-c0987334f793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:41.488980 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1437/2854] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:41.493546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc630c0d-65e7-4d32-aded-b7368889b61e admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.554922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc630c0d-65e7-4d32-aded-b7368889b61e admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.559601 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b954c35-2943-4ce5-b086-cbc9fa58c6f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.560209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b954c35-2943-4ce5-b086-cbc9fa58c6f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.560837 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b954c35-2943-4ce5-b086-cbc9fa58c6f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.560837 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b954c35-2943-4ce5-b086-cbc9fa58c6f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.561371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b954c35-2943-4ce5-b086-cbc9fa58c6f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.563293 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b954c35-2943-4ce5-b086-cbc9fa58c6f8 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.590428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1418/2855] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:26:41 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:41.594605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2452ef1-877b-4fea-856d-c0987334f793 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.606275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2452ef1-877b-4fea-856d-c0987334f793 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.632431 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:41.662981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64eba56-a41b-4b07-a54e-7f6a057e158d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.663551 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eba56-a41b-4b07-a54e-7f6a057e158d admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.663830 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eba56-a41b-4b07-a54e-7f6a057e158d admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.664101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64eba56-a41b-4b07-a54e-7f6a057e158d admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.664727 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64eba56-a41b-4b07-a54e-7f6a057e158d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.666902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64eba56-a41b-4b07-a54e-7f6a057e158d admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.688489 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1438/2856] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:26:41 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:41.692431 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b954c35-2943-4ce5-b086-cbc9fa58c6f8 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.703374 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b954c35-2943-4ce5-b086-cbc9fa58c6f8 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.708130 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fa7bc-d0c0-4088-b181-56a6e19c687c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.708728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fa7bc-d0c0-4088-b181-56a6e19c687c admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.709002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fa7bc-d0c0-4088-b181-56a6e19c687c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.709270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583fa7bc-d0c0-4088-b181-56a6e19c687c admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.709902 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583fa7bc-d0c0-4088-b181-56a6e19c687c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.711524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583fa7bc-d0c0-4088-b181-56a6e19c687c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.723337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1419/2857] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:26:41.727470 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64eba56-a41b-4b07-a54e-7f6a057e158d admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.738046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64eba56-a41b-4b07-a54e-7f6a057e158d admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.742560 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29f1d0-b28f-4302-aeb6-71bbcd18186c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.743193 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29f1d0-b28f-4302-aeb6-71bbcd18186c admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.743490 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29f1d0-b28f-4302-aeb6-71bbcd18186c admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.743763 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29f1d0-b28f-4302-aeb6-71bbcd18186c admin admin] PATH_INFO: `/v3/projects/714042de874a447598e0cea725b419ae/users/ba9af88de09042b4bb21cbf0dcc35182/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.758126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29f1d0-b28f-4302-aeb6-71bbcd18186c admin admin] RBAC: Authorizing `identity:create_grant(project_id=714042de874a447598e0cea725b419ae, user_id=ba9af88de09042b4bb21cbf0dcc35182, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.761666 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29f1d0-b28f-4302-aeb6-71bbcd18186c admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.773961 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1439/2858] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:26:41 2022] PUT /identity/v3/projects/714042de874a447598e0cea725b419ae/users/ba9af88de09042b4bb21cbf0dcc35182/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 10 14:26:41.778752 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fa7bc-d0c0-4088-b181-56a6e19c687c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.788885 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583fa7bc-d0c0-4088-b181-56a6e19c687c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.813507 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:41.850040 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85cae12-e861-444c-89bd-ae60ce0a9f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.850648 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cae12-e861-444c-89bd-ae60ce0a9f0f admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.850920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cae12-e861-444c-89bd-ae60ce0a9f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.851187 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85cae12-e861-444c-89bd-ae60ce0a9f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.851853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cae12-e861-444c-89bd-ae60ce0a9f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.853740 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85cae12-e861-444c-89bd-ae60ce0a9f0f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.865016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1420/2859] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:41.869533 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29f1d0-b28f-4302-aeb6-71bbcd18186c admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:41.884699 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29f1d0-b28f-4302-aeb6-71bbcd18186c admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:41.890038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec1384-618a-4e16-8642-0feb6933e702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:41.890691 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec1384-618a-4e16-8642-0feb6933e702 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.890996 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec1384-618a-4e16-8642-0feb6933e702 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.891269 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec1384-618a-4e16-8642-0feb6933e702 admin admin] PATH_INFO: `/v3/projects/714042de874a447598e0cea725b419ae/users/ba9af88de09042b4bb21cbf0dcc35182/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.893209 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec1384-618a-4e16-8642-0feb6933e702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=714042de874a447598e0cea725b419ae, user_id=ba9af88de09042b4bb21cbf0dcc35182, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:41.895902 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfec1384-618a-4e16-8642-0feb6933e702 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:41.916087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1440/2860] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:26:41 2022] PUT /identity/v3/projects/714042de874a447598e0cea725b419ae/users/ba9af88de09042b4bb21cbf0dcc35182/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:41.917721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331c50f-ad77-45ff-b074-65dbabe3d0a5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:41.918042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331c50f-ad77-45ff-b074-65dbabe3d0a5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:41.918368 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7331c50f-ad77-45ff-b074-65dbabe3d0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:41.953121 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-7331c50f-ad77-45ff-b074-65dbabe3d0a5 None None] MFA Rules not processed for user `ba9af88de09042b4bb21cbf0dcc35182`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:41.954346 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-7331c50f-ad77-45ff-b074-65dbabe3d0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:42.021880 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1421/2861] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:26:42.033667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec1384-618a-4e16-8642-0feb6933e702 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:42.035381 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-dfec1384-618a-4e16-8642-0feb6933e702 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:42.097385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec1384-618a-4e16-8642-0feb6933e702 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:42.101029 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaf7b5-9334-4590-9544-0b6cd410ce55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:42.101625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaf7b5-9334-4590-9544-0b6cd410ce55 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:42.101914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaf7b5-9334-4590-9544-0b6cd410ce55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:42.102167 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eaf7b5-9334-4590-9544-0b6cd410ce55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:42.111724 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eaf7b5-9334-4590-9544-0b6cd410ce55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:42.113412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eaf7b5-9334-4590-9544-0b6cd410ce55 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:42.122965 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1441/2862] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:42.205416 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331c50f-ad77-45ff-b074-65dbabe3d0a5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:42.216351 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7331c50f-ad77-45ff-b074-65dbabe3d0a5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:42.221551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8710e93a-44bf-4725-b45e-894fc7a4a6fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:42.222154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e93a-44bf-4725-b45e-894fc7a4a6fc service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:42.222474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e93a-44bf-4725-b45e-894fc7a4a6fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:42.222704 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8710e93a-44bf-4725-b45e-894fc7a4a6fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:42.286723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710e93a-44bf-4725-b45e-894fc7a4a6fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:42.288278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8710e93a-44bf-4725-b45e-894fc7a4a6fc service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:42.298938 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1422/2863] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:44.089813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaf7b5-9334-4590-9544-0b6cd410ce55 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:44.091430 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e3eaf7b5-9334-4590-9544-0b6cd410ce55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:44.155789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eaf7b5-9334-4590-9544-0b6cd410ce55 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:44.160243 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:44.160854 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:44.161122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:44.161387 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] PATH_INFO: `/v3/users/b3b88152b4e842b58bda0a2b5c9c886f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:44.162472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3b88152b4e842b58bda0a2b5c9c886f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:44.164419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:44.192003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:44.211495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3b88152b4e842b58bda0a2b5c9c886f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:44.218596 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3b88152b4e842b58bda0a2b5c9c886f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:44.219212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3b88152b4e842b58bda0a2b5c9c886f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:44.219867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3b88152b4e842b58bda0a2b5c9c886f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:44.226782 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3b88152b4e842b58bda0a2b5c9c886f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:44.244781 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1442/2864] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:26:44 2022] DELETE /identity/v3/users/b3b88152b4e842b58bda0a2b5c9c886f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:44.250088 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf4fad-7f97-410d-8d0d-206a1f35f6f3 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:44.250596 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf4fad-7f97-410d-8d0d-206a1f35f6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:44.250702 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cf4fad-7f97-410d-8d0d-206a1f35f6f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:44.270661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-c4cf4fad-7f97-410d-8d0d-206a1f35f6f3 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:44.271893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c4cf4fad-7f97-410d-8d0d-206a1f35f6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:44.312905 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:44.330793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1423/2865] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:44.341097 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:44.342504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:44.399276 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384fa51a-ef4a-439b-98aa-9438549700fa admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:44.403682 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476478d5-68ea-4975-9aec-74569b2cde93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:44.404275 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476478d5-68ea-4975-9aec-74569b2cde93 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:44.404546 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476478d5-68ea-4975-9aec-74569b2cde93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:44.404810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476478d5-68ea-4975-9aec-74569b2cde93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:44.413443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476478d5-68ea-4975-9aec-74569b2cde93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:44.414855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476478d5-68ea-4975-9aec-74569b2cde93 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:44.424968 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1443/2866] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:26:44.709063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf4fad-7f97-410d-8d0d-206a1f35f6f3 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:44.714379 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c4cf4fad-7f97-410d-8d0d-206a1f35f6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:44.733027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cf4fad-7f97-410d-8d0d-206a1f35f6f3 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:44.743220 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:44.743892 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:44.744236 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:44.744572 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] PATH_INFO: `/v3/projects/f2a216efa4a64d87bd5136b8e8071f37` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:44.746320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2a216efa4a64d87bd5136b8e8071f37)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:44.748290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:44.794725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2a216efa4a64d87bd5136b8e8071f37', 'request_id': 'req-51fe4614-c3c5-41b1-8818-f95be100e270'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:44.802768 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2a216efa4a64d87bd5136b8e8071f37', 'request_id': 'req-51fe4614-c3c5-41b1-8818-f95be100e270'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:44.808882 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2a216efa4a64d87bd5136b8e8071f37', 'request_id': 'req-51fe4614-c3c5-41b1-8818-f95be100e270'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:44.819672 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] The token cache is being invalidate because project f2a216efa4a64d87bd5136b8e8071f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:26:44.819958 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:44.820580 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-51fe4614-c3c5-41b1-8818-f95be100e270 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:44.822049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1424/2867] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:26:44 2022] DELETE /identity/v3/projects/f2a216efa4a64d87bd5136b8e8071f37 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:44.827015 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220901a8-5a89-4a5f-8955-939bb98a3176 None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:44.827336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220901a8-5a89-4a5f-8955-939bb98a3176 None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:44.827636 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220901a8-5a89-4a5f-8955-939bb98a3176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:44.847038 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-220901a8-5a89-4a5f-8955-939bb98a3176 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:44.848249 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-220901a8-5a89-4a5f-8955-939bb98a3176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:44.907790 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1444/2868] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:44.915657 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a308538-b27b-4e11-ae0e-4d69f848d049 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:44.915940 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a308538-b27b-4e11-ae0e-4d69f848d049 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:44.916209 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a308538-b27b-4e11-ae0e-4d69f848d049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:44.940278 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-5a308538-b27b-4e11-ae0e-4d69f848d049 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:44.941474 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-5a308538-b27b-4e11-ae0e-4d69f848d049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:44.949910 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1425/2869] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:44.954274 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220901a8-5a89-4a5f-8955-939bb98a3176 None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:44.964208 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220901a8-5a89-4a5f-8955-939bb98a3176 None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:44.969981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91662f93-daae-4f5a-ae31-1cab3712ca01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:44.970582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91662f93-daae-4f5a-ae31-1cab3712ca01 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:44.970857 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91662f93-daae-4f5a-ae31-1cab3712ca01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:44.971127 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91662f93-daae-4f5a-ae31-1cab3712ca01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:44.971860 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91662f93-daae-4f5a-ae31-1cab3712ca01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:26:44.972091 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91662f93-daae-4f5a-ae31-1cab3712ca01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:44.973672 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91662f93-daae-4f5a-ae31-1cab3712ca01 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:44.986062 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1445/2870] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:26:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:26:44.990554 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a308538-b27b-4e11-ae0e-4d69f848d049 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:45.000551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a308538-b27b-4e11-ae0e-4d69f848d049 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:45.005199 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff83fa1-fa13-4611-a9a8-57d3c8fef6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:45.005793 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83fa1-fa13-4611-a9a8-57d3c8fef6ec admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:45.006067 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83fa1-fa13-4611-a9a8-57d3c8fef6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:45.006337 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff83fa1-fa13-4611-a9a8-57d3c8fef6ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:45.006972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff83fa1-fa13-4611-a9a8-57d3c8fef6ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:45.008911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff83fa1-fa13-4611-a9a8-57d3c8fef6ec admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:45.037077 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1426/2871] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:26:44 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:45.040809 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91662f93-daae-4f5a-ae31-1cab3712ca01 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:45.051519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91662f93-daae-4f5a-ae31-1cab3712ca01 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:45.082914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:45.116845 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf5572-4e7b-49a4-a910-95817c1c51c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:45.117522 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf5572-4e7b-49a4-a910-95817c1c51c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:45.117799 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf5572-4e7b-49a4-a910-95817c1c51c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:45.118070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf5572-4e7b-49a4-a910-95817c1c51c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:45.118694 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf5572-4e7b-49a4-a910-95817c1c51c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:45.120788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bf5572-4e7b-49a4-a910-95817c1c51c9 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:45.145170 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1446/2872] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:26:45 2022] POST /identity/v3/users => generated 282 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:45.146350 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff83fa1-fa13-4611-a9a8-57d3c8fef6ec admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:45.164320 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff83fa1-fa13-4611-a9a8-57d3c8fef6ec admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:45.169347 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f83796-a6da-4fc0-8f10-bf344d15abc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:45.170275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f83796-a6da-4fc0-8f10-bf344d15abc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:45.170558 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f83796-a6da-4fc0-8f10-bf344d15abc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:45.171556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f83796-a6da-4fc0-8f10-bf344d15abc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:45.172216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f83796-a6da-4fc0-8f10-bf344d15abc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:45.173826 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f83796-a6da-4fc0-8f10-bf344d15abc2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:45.187268 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1427/2873] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:45.188472 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf5572-4e7b-49a4-a910-95817c1c51c9 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:45.198901 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bf5572-4e7b-49a4-a910-95817c1c51c9 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:45.203647 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef923778-5c23-40cc-af97-f03271feb663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:45.204273 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923778-5c23-40cc-af97-f03271feb663 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:45.204554 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923778-5c23-40cc-af97-f03271feb663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:45.204815 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef923778-5c23-40cc-af97-f03271feb663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:45.205449 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef923778-5c23-40cc-af97-f03271feb663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:45.207046 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef923778-5c23-40cc-af97-f03271feb663 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:45.221496 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1447/2874] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:45.223172 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f83796-a6da-4fc0-8f10-bf344d15abc2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:45.233814 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f83796-a6da-4fc0-8f10-bf344d15abc2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:45.238725 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fee1c-9bfa-40e2-8814-21cfc4a4fa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:45.239378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fee1c-9bfa-40e2-8814-21cfc4a4fa8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:45.239748 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fee1c-9bfa-40e2-8814-21cfc4a4fa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:45.240037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fee1c-9bfa-40e2-8814-21cfc4a4fa8d admin admin] PATH_INFO: `/v3/projects/4dd352c57b104e998c3cc182872add8f/users/b3ea9a7cbf7c4ae7bc9ceedabc2e6386/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:45.256462 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fee1c-9bfa-40e2-8814-21cfc4a4fa8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dd352c57b104e998c3cc182872add8f, user_id=b3ea9a7cbf7c4ae7bc9ceedabc2e6386, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:45.259311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fee1c-9bfa-40e2-8814-21cfc4a4fa8d admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:45.273823 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1428/2875] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:26:45 2022] PUT /identity/v3/projects/4dd352c57b104e998c3cc182872add8f/users/b3ea9a7cbf7c4ae7bc9ceedabc2e6386/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:45.285234 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187c632-9aa5-4108-be65-4e6cc4785f5b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:45.285519 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187c632-9aa5-4108-be65-4e6cc4785f5b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:45.285796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187c632-9aa5-4108-be65-4e6cc4785f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:45.313911 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-4187c632-9aa5-4108-be65-4e6cc4785f5b None None] MFA Rules not processed for user `b3ea9a7cbf7c4ae7bc9ceedabc2e6386`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:45.315155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4187c632-9aa5-4108-be65-4e6cc4785f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:45.386318 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1448/2876] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:45.397506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fee1c-9bfa-40e2-8814-21cfc4a4fa8d admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:45.399405 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-fd7fee1c-9bfa-40e2-8814-21cfc4a4fa8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:45.464222 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fee1c-9bfa-40e2-8814-21cfc4a4fa8d admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:45.468036 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac65672-5377-470f-afec-5ae79b6eead0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:45.468638 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac65672-5377-470f-afec-5ae79b6eead0 service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:45.468913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac65672-5377-470f-afec-5ae79b6eead0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:45.469531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac65672-5377-470f-afec-5ae79b6eead0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:45.478628 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac65672-5377-470f-afec-5ae79b6eead0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:45.481213 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac65672-5377-470f-afec-5ae79b6eead0 service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:45.491200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1429/2877] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:47.111814 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187c632-9aa5-4108-be65-4e6cc4785f5b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:47.113450 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-4187c632-9aa5-4108-be65-4e6cc4785f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:47.190114 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187c632-9aa5-4108-be65-4e6cc4785f5b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:47.194816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:47.195468 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:47.195755 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:47.196089 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] PATH_INFO: `/v3/users/10888d2d2b004e18b85e9f28f22b5daf` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:47.197268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10888d2d2b004e18b85e9f28f22b5daf)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:47.199244 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:47.248969 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:47.252446 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10888d2d2b004e18b85e9f28f22b5daf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:47.261747 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10888d2d2b004e18b85e9f28f22b5daf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:47.262872 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10888d2d2b004e18b85e9f28f22b5daf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:47.263635 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10888d2d2b004e18b85e9f28f22b5daf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:47.270932 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10888d2d2b004e18b85e9f28f22b5daf'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:47.290855 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1449/2878] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:26:47 2022] DELETE /identity/v3/users/10888d2d2b004e18b85e9f28f22b5daf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:47.296728 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ce6f-b95d-40a9-a902-044872ffc42b None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:47.297208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ce6f-b95d-40a9-a902-044872ffc42b None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:47.297364 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6671ce6f-b95d-40a9-a902-044872ffc42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:47.318272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-6671ce6f-b95d-40a9-a902-044872ffc42b None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:47.319809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6671ce6f-b95d-40a9-a902-044872ffc42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:47.323860 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:47.402794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1430/2879] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:47.412925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:47.414565 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:47.484956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee26ec79-27c9-4b05-b6de-14922a2f7e54 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:47.488765 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301e84c-2257-4098-823c-00e2db059ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:47.489363 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301e84c-2257-4098-823c-00e2db059ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:47.489638 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301e84c-2257-4098-823c-00e2db059ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:47.489906 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301e84c-2257-4098-823c-00e2db059ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:47.498766 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301e84c-2257-4098-823c-00e2db059ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:47.500224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301e84c-2257-4098-823c-00e2db059ce1 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:47.509408 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1450/2880] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:47.781306 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671ce6f-b95d-40a9-a902-044872ffc42b None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:47.783010 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6671ce6f-b95d-40a9-a902-044872ffc42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:47.796721 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6671ce6f-b95d-40a9-a902-044872ffc42b None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:47.801851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:47.802585 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:47.802864 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:47.803132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] PATH_INFO: `/v3/projects/85faa527e9cd4d19ba25cbafea6fa355` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:47.804290 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85faa527e9cd4d19ba25cbafea6fa355)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:47.806275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:47.853871 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85faa527e9cd4d19ba25cbafea6fa355', 'request_id': 'req-9a881524-5cf1-41c7-88d9-716cd8f3f213'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:47.861553 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85faa527e9cd4d19ba25cbafea6fa355', 'request_id': 'req-9a881524-5cf1-41c7-88d9-716cd8f3f213'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:47.867242 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85faa527e9cd4d19ba25cbafea6fa355', 'request_id': 'req-9a881524-5cf1-41c7-88d9-716cd8f3f213'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:47.878122 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] The token cache is being invalidate because project 85faa527e9cd4d19ba25cbafea6fa355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:26:47.878405 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:47.879073 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:47.880599 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1431/2881] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:26:47 2022] DELETE /identity/v3/projects/85faa527e9cd4d19ba25cbafea6fa355 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:47.884611 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301e84c-2257-4098-823c-00e2db059ce1 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:47.894956 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301e84c-2257-4098-823c-00e2db059ce1 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:47.951890 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:47.952502 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:47.952796 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:47.953042 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] PATH_INFO: `/v3/users/ba9af88de09042b4bb21cbf0dcc35182` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:47.954314 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba9af88de09042b4bb21cbf0dcc35182)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:47.956259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:48.003703 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba9af88de09042b4bb21cbf0dcc35182'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:48.012573 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba9af88de09042b4bb21cbf0dcc35182'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:48.013303 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba9af88de09042b4bb21cbf0dcc35182'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:48.013974 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba9af88de09042b4bb21cbf0dcc35182'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:48.021858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-dfc235f1-0b4d-46b8-8f61-6c466fce68a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba9af88de09042b4bb21cbf0dcc35182'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:48.023983 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:48.041816 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1451/2882] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:26:47 2022] DELETE /identity/v3/users/ba9af88de09042b4bb21cbf0dcc35182 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:48.316119 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:48.317718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:48.340047 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a881524-5cf1-41c7-88d9-716cd8f3f213 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:48.389395 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:48.390027 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:48.390304 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:48.390576 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] PATH_INFO: `/v3/projects/714042de874a447598e0cea725b419ae` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:48.391710 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=714042de874a447598e0cea725b419ae)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:48.393642 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:48.433301 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:48.449529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '714042de874a447598e0cea725b419ae', 'request_id': 'req-e668d6da-ca58-4c46-a9bc-ca064565e5e5'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:48.467644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '714042de874a447598e0cea725b419ae', 'request_id': 'req-e668d6da-ca58-4c46-a9bc-ca064565e5e5'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:48.467644 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '714042de874a447598e0cea725b419ae', 'request_id': 'req-e668d6da-ca58-4c46-a9bc-ca064565e5e5'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:48.475151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] The token cache is being invalidate because project 714042de874a447598e0cea725b419ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:26:48.475432 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:48.476102 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-e668d6da-ca58-4c46-a9bc-ca064565e5e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:48.477606 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1432/2883] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:26:48 2022] DELETE /identity/v3/projects/714042de874a447598e0cea725b419ae => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:48.484681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cac21-3ebe-43df-9e9d-9c65ceace5ce None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:48.485545 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cac21-3ebe-43df-9e9d-9c65ceace5ce None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:48.485817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cac21-3ebe-43df-9e9d-9c65ceace5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:48.506797 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-d05cac21-3ebe-43df-9e9d-9c65ceace5ce None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:48.507938 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-d05cac21-3ebe-43df-9e9d-9c65ceace5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:48.577159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1452/2884] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:48.588954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5af1ce-c36c-4c6a-af2d-e295e2dd8fc3 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:48.588954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5af1ce-c36c-4c6a-af2d-e295e2dd8fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:48.588954 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5af1ce-c36c-4c6a-af2d-e295e2dd8fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:48.613378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-eb5af1ce-c36c-4c6a-af2d-e295e2dd8fc3 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:48.616508 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-eb5af1ce-c36c-4c6a-af2d-e295e2dd8fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:48.629002 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1433/2885] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:26:48.633186 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cac21-3ebe-43df-9e9d-9c65ceace5ce None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:48.643991 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cac21-3ebe-43df-9e9d-9c65ceace5ce None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:48.649277 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97bc22-7c82-40cf-ac82-788768925cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:48.649883 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97bc22-7c82-40cf-ac82-788768925cb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:48.650176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97bc22-7c82-40cf-ac82-788768925cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:48.650419 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97bc22-7c82-40cf-ac82-788768925cb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:48.651664 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97bc22-7c82-40cf-ac82-788768925cb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81589) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 10 14:26:48.652033 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97bc22-7c82-40cf-ac82-788768925cb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:48.653632 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a97bc22-7c82-40cf-ac82-788768925cb0 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:48.665599 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1453/2886] 10.222.0.26 () {58 vars in 1153 bytes} [Sat Sep 10 14:26:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 10 14:26:48.670188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5af1ce-c36c-4c6a-af2d-e295e2dd8fc3 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:48.681484 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5af1ce-c36c-4c6a-af2d-e295e2dd8fc3 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:48.686397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01e44d-db80-4136-b4b4-07f47fdcb447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:48.686995 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01e44d-db80-4136-b4b4-07f47fdcb447 admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:48.687272 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01e44d-db80-4136-b4b4-07f47fdcb447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:48.687629 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01e44d-db80-4136-b4b4-07f47fdcb447 admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:48.688363 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01e44d-db80-4136-b4b4-07f47fdcb447 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:48.690261 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01e44d-db80-4136-b4b4-07f47fdcb447 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:48.719608 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1434/2887] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:26:48 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:48.723810 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97bc22-7c82-40cf-ac82-788768925cb0 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:48.734037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97bc22-7c82-40cf-ac82-788768925cb0 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:48.795716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192b7fb-1ee3-42ad-b13a-3b61f400bac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:48.796343 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192b7fb-1ee3-42ad-b13a-3b61f400bac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:48.796598 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192b7fb-1ee3-42ad-b13a-3b61f400bac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:48.796867 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192b7fb-1ee3-42ad-b13a-3b61f400bac2 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:48.797541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192b7fb-1ee3-42ad-b13a-3b61f400bac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:48.799681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192b7fb-1ee3-42ad-b13a-3b61f400bac2 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:48.824289 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1454/2888] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:26:48 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:48.828490 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01e44d-db80-4136-b4b4-07f47fdcb447 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:48.839865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01e44d-db80-4136-b4b4-07f47fdcb447 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:48.844711 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665ff2f-9c8d-49df-825f-a48c2ea532cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:48.845459 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ff2f-9c8d-49df-825f-a48c2ea532cc admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:48.845735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ff2f-9c8d-49df-825f-a48c2ea532cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:48.846156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665ff2f-9c8d-49df-825f-a48c2ea532cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:48.846929 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665ff2f-9c8d-49df-825f-a48c2ea532cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:48.848781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665ff2f-9c8d-49df-825f-a48c2ea532cc admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:48.861258 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1435/2889] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:48.865600 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192b7fb-1ee3-42ad-b13a-3b61f400bac2 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:48.877409 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192b7fb-1ee3-42ad-b13a-3b61f400bac2 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:48.882084 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e5130-abe9-4cdf-9bb3-9cb94ec73406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:48.882675 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e5130-abe9-4cdf-9bb3-9cb94ec73406 admin admin] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:48.882947 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e5130-abe9-4cdf-9bb3-9cb94ec73406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:48.883212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e5130-abe9-4cdf-9bb3-9cb94ec73406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:48.883886 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e5130-abe9-4cdf-9bb3-9cb94ec73406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:48.886156 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e5130-abe9-4cdf-9bb3-9cb94ec73406 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:48.900581 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1455/2890] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 10 14:26:48.902454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665ff2f-9c8d-49df-825f-a48c2ea532cc admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:48.912911 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665ff2f-9c8d-49df-825f-a48c2ea532cc admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:48.918879 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f34e4-f051-4be9-bf7f-f9af8b4f2b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:48.919574 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f34e4-f051-4be9-bf7f-f9af8b4f2b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:48.919849 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f34e4-f051-4be9-bf7f-f9af8b4f2b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:48.920115 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f34e4-f051-4be9-bf7f-f9af8b4f2b97 admin admin] PATH_INFO: `/v3/projects/8830af55af5f440e9f9be65da9d809d2/users/17897882329b45d8aed18cdbe87ef67f/roles/bb37d85c981a4f79b9602ba7d0bc6fa5` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:48.935979 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f34e4-f051-4be9-bf7f-f9af8b4f2b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8830af55af5f440e9f9be65da9d809d2, user_id=17897882329b45d8aed18cdbe87ef67f, role_id=bb37d85c981a4f79b9602ba7d0bc6fa5)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:48.938718 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f34e4-f051-4be9-bf7f-f9af8b4f2b97 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:48.952266 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1436/2891] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:26:48 2022] PUT /identity/v3/projects/8830af55af5f440e9f9be65da9d809d2/users/17897882329b45d8aed18cdbe87ef67f/roles/bb37d85c981a4f79b9602ba7d0bc6fa5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:48.962532 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14dccb-db9b-4bc2-bfe1-044a0cbb47dd None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:48.962812 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14dccb-db9b-4bc2-bfe1-044a0cbb47dd None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:48.963107 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b14dccb-db9b-4bc2-bfe1-044a0cbb47dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:48.990259 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-1b14dccb-db9b-4bc2-bfe1-044a0cbb47dd None None] MFA Rules not processed for user `17897882329b45d8aed18cdbe87ef67f`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:48.991494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1b14dccb-db9b-4bc2-bfe1-044a0cbb47dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:49.028438 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:49.055853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1456/2892] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:49.061014 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f34e4-f051-4be9-bf7f-f9af8b4f2b97 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:49.122183 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f34e4-f051-4be9-bf7f-f9af8b4f2b97 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:49.126680 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5c216-cfbf-4c87-873a-2bdfa69acbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:49.127273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5c216-cfbf-4c87-873a-2bdfa69acbdc admin admin] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:49.127579 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5c216-cfbf-4c87-873a-2bdfa69acbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:49.127851 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c5c216-cfbf-4c87-873a-2bdfa69acbdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:49.128472 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5c216-cfbf-4c87-873a-2bdfa69acbdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:49.130401 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c5c216-cfbf-4c87-873a-2bdfa69acbdc admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:49.157200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1437/2893] 10.222.0.26 () {60 vars in 1153 bytes} [Sat Sep 10 14:26:49 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:49.161262 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14dccb-db9b-4bc2-bfe1-044a0cbb47dd None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:49.172757 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b14dccb-db9b-4bc2-bfe1-044a0cbb47dd None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:49.227321 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28f99-8f07-4b12-b6db-1697184e0431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:49.227946 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28f99-8f07-4b12-b6db-1697184e0431 admin admin] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:49.228223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28f99-8f07-4b12-b6db-1697184e0431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:49.228495 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca28f99-8f07-4b12-b6db-1697184e0431 admin admin] PATH_INFO: `/v3/users` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:49.229644 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca28f99-8f07-4b12-b6db-1697184e0431 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:49.231681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca28f99-8f07-4b12-b6db-1697184e0431 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:49.252462 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1457/2894] 10.222.0.26 () {60 vars in 1144 bytes} [Sat Sep 10 14:26:49 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 10 14:26:49.256524 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5c216-cfbf-4c87-873a-2bdfa69acbdc admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:49.266770 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c5c216-cfbf-4c87-873a-2bdfa69acbdc admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:49.271378 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57116f-98ee-403c-b234-db2ea9f1a03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:49.272009 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57116f-98ee-403c-b234-db2ea9f1a03c admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:49.272282 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57116f-98ee-403c-b234-db2ea9f1a03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:49.272547 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57116f-98ee-403c-b234-db2ea9f1a03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:49.273175 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57116f-98ee-403c-b234-db2ea9f1a03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:49.274761 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57116f-98ee-403c-b234-db2ea9f1a03c admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:49.285156 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1438/2895] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:26:49.289247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28f99-8f07-4b12-b6db-1697184e0431 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:49.299475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca28f99-8f07-4b12-b6db-1697184e0431 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:49.303987 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e0713-db39-403d-9c0b-a3dc56075e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:49.304619 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e0713-db39-403d-9c0b-a3dc56075e03 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:49.304891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e0713-db39-403d-9c0b-a3dc56075e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:49.305158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e0713-db39-403d-9c0b-a3dc56075e03 admin admin] PATH_INFO: `/v3/projects/68f06c1beb54435bbd1a8f5bee137608/users/50b25173f70c4b9c8640c44a155358ab/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:49.319976 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e0713-db39-403d-9c0b-a3dc56075e03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f06c1beb54435bbd1a8f5bee137608, user_id=50b25173f70c4b9c8640c44a155358ab, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:49.322609 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e0713-db39-403d-9c0b-a3dc56075e03 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:49.335297 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1458/2896] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:26:49 2022] PUT /identity/v3/projects/68f06c1beb54435bbd1a8f5bee137608/users/50b25173f70c4b9c8640c44a155358ab/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:49.339397 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57116f-98ee-403c-b234-db2ea9f1a03c admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:49.350493 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57116f-98ee-403c-b234-db2ea9f1a03c admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:49.383239 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:49.410512 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b869b0b-e63e-47ce-9ad5-2ca309f549ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:49.411149 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b869b0b-e63e-47ce-9ad5-2ca309f549ef admin admin] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:49.411428 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b869b0b-e63e-47ce-9ad5-2ca309f549ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:49.411724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b869b0b-e63e-47ce-9ad5-2ca309f549ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:49.412361 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b869b0b-e63e-47ce-9ad5-2ca309f549ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:49.413965 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b869b0b-e63e-47ce-9ad5-2ca309f549ef admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:49.425184 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1439/2897] 10.222.0.26 () {58 vars in 1122 bytes} [Sat Sep 10 14:26:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 10 14:26:49.429399 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e0713-db39-403d-9c0b-a3dc56075e03 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:49.441224 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e0713-db39-403d-9c0b-a3dc56075e03 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:49.446212 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfa1c8-60a1-4110-a9d7-4f2a098e9849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:49.446776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfa1c8-60a1-4110-a9d7-4f2a098e9849 admin admin] REQUEST_METHOD: `PUT` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:49.447108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfa1c8-60a1-4110-a9d7-4f2a098e9849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:49.447535 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dfa1c8-60a1-4110-a9d7-4f2a098e9849 admin admin] PATH_INFO: `/v3/projects/68f06c1beb54435bbd1a8f5bee137608/users/50b25173f70c4b9c8640c44a155358ab/roles/5ef89d7e38744a2aa5e3bdf0511d87d6` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:49.449734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfa1c8-60a1-4110-a9d7-4f2a098e9849 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f06c1beb54435bbd1a8f5bee137608, user_id=50b25173f70c4b9c8640c44a155358ab, role_id=5ef89d7e38744a2aa5e3bdf0511d87d6)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:49.452481 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dfa1c8-60a1-4110-a9d7-4f2a098e9849 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:49.463345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1459/2898] 10.222.0.26 () {60 vars in 1483 bytes} [Sat Sep 10 14:26:49 2022] PUT /identity/v3/projects/68f06c1beb54435bbd1a8f5bee137608/users/50b25173f70c4b9c8640c44a155358ab/roles/5ef89d7e38744a2aa5e3bdf0511d87d6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:49.474753 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24102bb3-d9ea-47dd-a269-47e3b753819a None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:49.475112 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24102bb3-d9ea-47dd-a269-47e3b753819a None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:49.475496 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24102bb3-d9ea-47dd-a269-47e3b753819a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:49.505063 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-24102bb3-d9ea-47dd-a269-47e3b753819a None None] MFA Rules not processed for user `50b25173f70c4b9c8640c44a155358ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:49.506362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-24102bb3-d9ea-47dd-a269-47e3b753819a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:49.580790 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1440/2899] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:49 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 10 14:26:49.592925 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfa1c8-60a1-4110-a9d7-4f2a098e9849 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:49.594702 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e8dfa1c8-60a1-4110-a9d7-4f2a098e9849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:49.653051 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dfa1c8-60a1-4110-a9d7-4f2a098e9849 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:49.656552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef61057-1349-499a-ac82-8f471ce3fd83 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:49.657148 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef61057-1349-499a-ac82-8f471ce3fd83 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:49.657420 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef61057-1349-499a-ac82-8f471ce3fd83 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:49.657750 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef61057-1349-499a-ac82-8f471ce3fd83 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:49.668117 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef61057-1349-499a-ac82-8f471ce3fd83 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:49.669547 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef61057-1349-499a-ac82-8f471ce3fd83 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:49.679634 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1460/2900] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:49 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:49.762535 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24102bb3-d9ea-47dd-a269-47e3b753819a None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:49.773162 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24102bb3-d9ea-47dd-a269-47e3b753819a None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:49.777893 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dab29bf-df4a-4062-b4de-a6766b914a9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:49.778494 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab29bf-df4a-4062-b4de-a6766b914a9d service cinder] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:49.778786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab29bf-df4a-4062-b4de-a6766b914a9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:49.779085 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab29bf-df4a-4062-b4de-a6766b914a9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:49.838037 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab29bf-df4a-4062-b4de-a6766b914a9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:49.840388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dab29bf-df4a-4062-b4de-a6766b914a9d service cinder] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:49.849620 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1441/2901] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:50.145518 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef61057-1349-499a-ac82-8f471ce3fd83 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:50.183315 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:50.220157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef61057-1349-499a-ac82-8f471ce3fd83 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:50.225159 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:50.225459 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:50.225734 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:50.226002 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] PATH_INFO: `/v3/users/17897882329b45d8aed18cdbe87ef67f` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:50.227122 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17897882329b45d8aed18cdbe87ef67f)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:50.229806 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:50.282574 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17897882329b45d8aed18cdbe87ef67f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:50.293914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17897882329b45d8aed18cdbe87ef67f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:50.295232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17897882329b45d8aed18cdbe87ef67f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:50.295848 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17897882329b45d8aed18cdbe87ef67f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:50.303858 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17897882329b45d8aed18cdbe87ef67f'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:50.333012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1461/2902] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:26:50 2022] DELETE /identity/v3/users/17897882329b45d8aed18cdbe87ef67f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:50.337345 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f5c5c-b75e-4303-929d-7af7a4e2247f None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:50.337639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f5c5c-b75e-4303-929d-7af7a4e2247f None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:50.337922 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f5c5c-b75e-4303-929d-7af7a4e2247f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:50.357852 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-693f5c5c-b75e-4303-929d-7af7a4e2247f None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:26:50.359065 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-693f5c5c-b75e-4303-929d-7af7a4e2247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:50.408270 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:50.421499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1442/2903] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:26:50.431174 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:50.433426 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:50.491827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8bf92-f5f2-4d34-a295-7c8ba8169349 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:50.496120 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c0c45-059e-462a-a715-0438d0ea3534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:50.496726 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c0c45-059e-462a-a715-0438d0ea3534 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:50.497034 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c0c45-059e-462a-a715-0438d0ea3534 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:50.497306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c0c45-059e-462a-a715-0438d0ea3534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:50.506817 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c0c45-059e-462a-a715-0438d0ea3534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:50.508788 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c0c45-059e-462a-a715-0438d0ea3534 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:50.518708 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1462/2904] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:26:50.822004 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f5c5c-b75e-4303-929d-7af7a4e2247f None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:50.823530 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-693f5c5c-b75e-4303-929d-7af7a4e2247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:50.842000 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f5c5c-b75e-4303-929d-7af7a4e2247f None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:50.846966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:50.847820 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:50.848042 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:50.848244 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] PATH_INFO: `/v3/projects/8830af55af5f440e9f9be65da9d809d2` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:50.850439 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8830af55af5f440e9f9be65da9d809d2)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:50.852525 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:50.912033 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8830af55af5f440e9f9be65da9d809d2', 'request_id': 'req-a667ae38-110d-4420-9cb9-7e0f8c873325'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:50.923003 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8830af55af5f440e9f9be65da9d809d2', 'request_id': 'req-a667ae38-110d-4420-9cb9-7e0f8c873325'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:50.931430 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8830af55af5f440e9f9be65da9d809d2', 'request_id': 'req-a667ae38-110d-4420-9cb9-7e0f8c873325'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:50.946708 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] The token cache is being invalidate because project 8830af55af5f440e9f9be65da9d809d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:26:50.947001 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:50.948677 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:50.953906 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1443/2905] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:26:50 2022] DELETE /identity/v3/projects/8830af55af5f440e9f9be65da9d809d2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:50.955349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c0c45-059e-462a-a715-0438d0ea3534 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:50.966248 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c0c45-059e-462a-a715-0438d0ea3534 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:51.021345 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:51.031302 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:51.032525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:51.033112 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:51.033569 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] PATH_INFO: `/v3/users/50b25173f70c4b9c8640c44a155358ab` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:51.034928 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50b25173f70c4b9c8640c44a155358ab)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:51.037717 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:51.100841 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50b25173f70c4b9c8640c44a155358ab'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:51.110197 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50b25173f70c4b9c8640c44a155358ab'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:51.110931 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50b25173f70c4b9c8640c44a155358ab'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:51.111563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50b25173f70c4b9c8640c44a155358ab'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:51.120878 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50b25173f70c4b9c8640c44a155358ab'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:51.143475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1463/2906] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:26:50 2022] DELETE /identity/v3/users/50b25173f70c4b9c8640c44a155358ab => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:51.420531 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:51.421981 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:51.440365 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667ae38-110d-4420-9cb9-7e0f8c873325 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:51.499639 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:51.500080 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:51.500355 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:51.500624 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] PATH_INFO: `/v3/projects/68f06c1beb54435bbd1a8f5bee137608` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:51.501720 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f06c1beb54435bbd1a8f5bee137608)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:51.503645 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:51.550049 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f06c1beb54435bbd1a8f5bee137608', 'request_id': 'req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:51.557312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f06c1beb54435bbd1a8f5bee137608', 'request_id': 'req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:51.563201 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f06c1beb54435bbd1a8f5bee137608', 'request_id': 'req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:51.574563 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] The token cache is being invalidate because project 68f06c1beb54435bbd1a8f5bee137608 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:26:51.574846 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:51.575556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:26:51.577007 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1444/2907] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:26:51 2022] DELETE /identity/v3/projects/68f06c1beb54435bbd1a8f5bee137608 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:26:52.231773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:26:52.233454 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:52.321674 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d324107-5022-400b-b20c-c6a6d82df046 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:26:52.387564 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb929426-2835-4da0-b3b4-a3942aa7298e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:26:52.388176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb929426-2835-4da0-b3b4-a3942aa7298e service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:26:52.388478 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb929426-2835-4da0-b3b4-a3942aa7298e service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:26:52.388722 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb929426-2835-4da0-b3b4-a3942aa7298e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:26:52.390494 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cb929426-2835-4da0-b3b4-a3942aa7298e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:26:52.402927 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb929426-2835-4da0-b3b4-a3942aa7298e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:26:52.404311 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb929426-2835-4da0-b3b4-a3942aa7298e service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:26:52.409241 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:26:52.418109 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1464/2908] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:26:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:27:01.570786 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=81,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:27:02.421441 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:27:14.560390 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:27:14.561809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:27:14.630774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbfe91b-e32c-4cfd-8b4f-c9253cf1d1f2 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:27:14.636568 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e81300-4c32-4129-a085-e999a5ae5306 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:27:14.637180 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e81300-4c32-4129-a085-e999a5ae5306 service placement] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:27:14.637461 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e81300-4c32-4129-a085-e999a5ae5306 service placement] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:27:14.637733 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e81300-4c32-4129-a085-e999a5ae5306 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:27:14.639529 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-80e81300-4c32-4129-a085-e999a5ae5306 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:27:14.651178 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e81300-4c32-4129-a085-e999a5ae5306 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:27:14.652805 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e81300-4c32-4129-a085-e999a5ae5306 service placement] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:27:14.663086 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1445/2909] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:27:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:27:16.166058 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb929426-2835-4da0-b3b4-a3942aa7298e service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:27:16.167552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-cb929426-2835-4da0-b3b4-a3942aa7298e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:27:16.230216 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb929426-2835-4da0-b3b4-a3942aa7298e service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:27:16.234323 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bf3e9-a324-4e7b-b9d0-8526b5540f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:27:16.234999 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bf3e9-a324-4e7b-b9d0-8526b5540f91 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:27:16.235370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bf3e9-a324-4e7b-b9d0-8526b5540f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:27:16.235736 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152bf3e9-a324-4e7b-b9d0-8526b5540f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:27:16.237789 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-152bf3e9-a324-4e7b-b9d0-8526b5540f91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:27:16.252325 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bf3e9-a324-4e7b-b9d0-8526b5540f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:27:16.254511 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152bf3e9-a324-4e7b-b9d0-8526b5540f91 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:27:16.264905 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1465/2910] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:27:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:27:16.354723 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e81300-4c32-4129-a085-e999a5ae5306 service placement] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:27:16.364784 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e81300-4c32-4129-a085-e999a5ae5306 service placement] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:27:16.368709 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcf4c1-c013-463f-a2ac-d9443407767d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:27:16.369312 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcf4c1-c013-463f-a2ac-d9443407767d service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:27:16.369594 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcf4c1-c013-463f-a2ac-d9443407767d service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:27:16.369868 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcf4c1-c013-463f-a2ac-d9443407767d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:27:16.371671 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ddbcf4c1-c013-463f-a2ac-d9443407767d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:27:16.387076 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcf4c1-c013-463f-a2ac-d9443407767d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:27:16.388758 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcf4c1-c013-463f-a2ac-d9443407767d service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:27:16.400970 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1446/2911] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:27:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:27:26.260737 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=26 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:27:26.395077 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=38 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:27:28.529436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bf3e9-a324-4e7b-b9d0-8526b5540f91 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:27:28.530943 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-152bf3e9-a324-4e7b-b9d0-8526b5540f91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:27:28.595732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152bf3e9-a324-4e7b-b9d0-8526b5540f91 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:27:28.600268 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:27:28.600914 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:27:28.601190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:27:28.601460 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] PATH_INFO: `/v3/users/b3ea9a7cbf7c4ae7bc9ceedabc2e6386` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:27:28.603190 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3ea9a7cbf7c4ae7bc9ceedabc2e6386)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:27:28.605138 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:27:28.656605 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3ea9a7cbf7c4ae7bc9ceedabc2e6386'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:27:28.664537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3ea9a7cbf7c4ae7bc9ceedabc2e6386'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:27:28.665978 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3ea9a7cbf7c4ae7bc9ceedabc2e6386'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:27:28.666615 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3ea9a7cbf7c4ae7bc9ceedabc2e6386'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:27:28.674432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3ea9a7cbf7c4ae7bc9ceedabc2e6386'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:27:28.691653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1466/2912] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:27:28 2022] DELETE /identity/v3/users/b3ea9a7cbf7c4ae7bc9ceedabc2e6386 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:27:28.696200 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfb0f5-4ea4-4f3c-aecf-852dc06cb0f5 None None] REQUEST_METHOD: `POST` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:27:28.696774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfb0f5-4ea4-4f3c-aecf-852dc06cb0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:27:28.696774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacfb0f5-4ea4-4f3c-aecf-852dc06cb0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:27:28.722024 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.auth.core [None req-bacfb0f5-4ea4-4f3c-aecf-852dc06cb0f5 None None] MFA Rules not processed for user `25a17c1158c94c229e16c77c070bd9d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81588) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:27:28.723418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bacfb0f5-4ea4-4f3c-aecf-852dc06cb0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:27:28.786707 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1447/2913] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:27:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:27:28.797145 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:27:28.798557 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:27:28.856516 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb49ab9-08e0-4afb-9f9d-5933a27bdbe3 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:27:28.860176 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304a986-d88a-4e53-8141-7d042a5bb7c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:27:28.860776 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304a986-d88a-4e53-8141-7d042a5bb7c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:27:28.861056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304a986-d88a-4e53-8141-7d042a5bb7c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:27:28.861324 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304a986-d88a-4e53-8141-7d042a5bb7c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:27:28.870711 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304a986-d88a-4e53-8141-7d042a5bb7c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:27:28.872185 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c304a986-d88a-4e53-8141-7d042a5bb7c6 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:27:28.882151 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1467/2914] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:27:29.162972 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfb0f5-4ea4-4f3c-aecf-852dc06cb0f5 None None] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:27:29.164569 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-bacfb0f5-4ea4-4f3c-aecf-852dc06cb0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:27:29.177408 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacfb0f5-4ea4-4f3c-aecf-852dc06cb0f5 None None] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:27:29.182408 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:27:29.183048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:27:29.183828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:27:29.183828 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] PATH_INFO: `/v3/projects/4dd352c57b104e998c3cc182872add8f` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:27:29.184787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dd352c57b104e998c3cc182872add8f)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:27:29.186724 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:27:29.238516 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dd352c57b104e998c3cc182872add8f', 'request_id': 'req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:27:29.246302 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dd352c57b104e998c3cc182872add8f', 'request_id': 'req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:27:29.254993 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dd352c57b104e998c3cc182872add8f', 'request_id': 'req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:27:29.268216 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] The token cache is being invalidate because project 4dd352c57b104e998c3cc182872add8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:27:29.268505 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:27:29.269188 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:27:29.270719 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1448/2915] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:27:29 2022] DELETE /identity/v3/projects/4dd352c57b104e998c3cc182872add8f => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:27:38.877549 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:27:39.264789 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:28:58.473227 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304a986-d88a-4e53-8141-7d042a5bb7c6 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:28:58.482563 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304a986-d88a-4e53-8141-7d042a5bb7c6 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:28:58.530670 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f2346-03a7-48a4-9b96-c0ea9a57994a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:28:58.531257 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2346-03a7-48a4-9b96-c0ea9a57994a service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:28:58.531555 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2346-03a7-48a4-9b96-c0ea9a57994a service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:28:58.531824 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8f2346-03a7-48a4-9b96-c0ea9a57994a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:28:58.533618 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6b8f2346-03a7-48a4-9b96-c0ea9a57994a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:28:58.590531 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f2346-03a7-48a4-9b96-c0ea9a57994a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:28:58.591981 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8f2346-03a7-48a4-9b96-c0ea9a57994a service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:28:58.600620 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1468/2916] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:28:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:28:59.629243 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:28:59.630862 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:28:59.643986 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f52f8-5a07-42d0-b6d2-9d3e5e3cd458 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:28:59.647587 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d357c8-feb3-457e-a28d-7a0c02c13a5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:28:59.648186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d357c8-feb3-457e-a28d-7a0c02c13a5b service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:28:59.648467 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d357c8-feb3-457e-a28d-7a0c02c13a5b service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:28:59.648739 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d357c8-feb3-457e-a28d-7a0c02c13a5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:28:59.650454 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-41d357c8-feb3-457e-a28d-7a0c02c13a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:28:59.724499 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d357c8-feb3-457e-a28d-7a0c02c13a5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:28:59.725942 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d357c8-feb3-457e-a28d-7a0c02c13a5b service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:28:59.737683 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1449/2917] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:28:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:29:00.930586 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f2346-03a7-48a4-9b96-c0ea9a57994a service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:00.933108 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-6b8f2346-03a7-48a4-9b96-c0ea9a57994a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:00.948370 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8f2346-03a7-48a4-9b96-c0ea9a57994a service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:00.953231 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42e4a2-0409-4882-8533-c6e36eda6c2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:00.953885 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42e4a2-0409-4882-8533-c6e36eda6c2a service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:00.954184 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42e4a2-0409-4882-8533-c6e36eda6c2a service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:00.954450 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42e4a2-0409-4882-8533-c6e36eda6c2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:00.956308 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1e42e4a2-0409-4882-8533-c6e36eda6c2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:00.975043 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42e4a2-0409-4882-8533-c6e36eda6c2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:00.976528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42e4a2-0409-4882-8533-c6e36eda6c2a service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:00.992335 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1469/2918] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:29:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:29:01.912964 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d357c8-feb3-457e-a28d-7a0c02c13a5b service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:01.914556 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-41d357c8-feb3-457e-a28d-7a0c02c13a5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:01.929084 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d357c8-feb3-457e-a28d-7a0c02c13a5b service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:01.936241 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0bd7e-54e3-4e91-9063-6f8dfed188a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:01.936891 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0bd7e-54e3-4e91-9063-6f8dfed188a6 service nova] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:01.937171 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0bd7e-54e3-4e91-9063-6f8dfed188a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:01.937566 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a0bd7e-54e3-4e91-9063-6f8dfed188a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:01.939652 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-a2a0bd7e-54e3-4e91-9063-6f8dfed188a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:01.955688 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0bd7e-54e3-4e91-9063-6f8dfed188a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:01.957433 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a0bd7e-54e3-4e91-9063-6f8dfed188a6 service nova] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:01.969716 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1450/2919] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:29:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:29:02.490450 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42e4a2-0409-4882-8533-c6e36eda6c2a service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:02.492003 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-1e42e4a2-0409-4882-8533-c6e36eda6c2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:02.551773 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42e4a2-0409-4882-8533-c6e36eda6c2a service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:02.555624 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12291bf-9794-46bb-9de7-1980bf5cc7d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdc278f50e794e12a27d30fca9d131c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:02.556222 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12291bf-9794-46bb-9de7-1980bf5cc7d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:02.556498 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12291bf-9794-46bb-9de7-1980bf5cc7d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:02.556795 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12291bf-9794-46bb-9de7-1980bf5cc7d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:02.558536 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b12291bf-9794-46bb-9de7-1980bf5cc7d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:02.615326 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=100 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:29:02.617818 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12291bf-9794-46bb-9de7-1980bf5cc7d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:02.620507 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12291bf-9794-46bb-9de7-1980bf5cc7d1 service cinder] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:02.629802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1470/2920] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:29:02 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:29:03.519847 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0bd7e-54e3-4e91-9063-6f8dfed188a6 service nova] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:03.529362 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a0bd7e-54e3-4e91-9063-6f8dfed188a6 service nova] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:03.532825 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca650001-73c1-478e-b420-238dde9e419c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:03.533418 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca650001-73c1-478e-b420-238dde9e419c service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:03.533696 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca650001-73c1-478e-b420-238dde9e419c service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:03.533966 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca650001-73c1-478e-b420-238dde9e419c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:03.535689 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ca650001-73c1-478e-b420-238dde9e419c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:03.598388 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca650001-73c1-478e-b420-238dde9e419c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:03.599866 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca650001-73c1-478e-b420-238dde9e419c service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:03.609429 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1451/2921] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:29:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:29:06.475087 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12291bf-9794-46bb-9de7-1980bf5cc7d1 service cinder] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:06.476719 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-b12291bf-9794-46bb-9de7-1980bf5cc7d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:06.572793 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12291bf-9794-46bb-9de7-1980bf5cc7d1 service cinder] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:06.578432 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b5acc-dfd6-43e2-bc94-4d117d80fbf4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:06.579037 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b5acc-dfd6-43e2-bc94-4d117d80fbf4 service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:06.579313 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b5acc-dfd6-43e2-bc94-4d117d80fbf4 service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:06.579616 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013b5acc-dfd6-43e2-bc94-4d117d80fbf4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:06.581464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-013b5acc-dfd6-43e2-bc94-4d117d80fbf4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:06.593585 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b5acc-dfd6-43e2-bc94-4d117d80fbf4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:06.595286 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013b5acc-dfd6-43e2-bc94-4d117d80fbf4 service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:06.606290 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1471/2922] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:29:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:29:09.689634 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca650001-73c1-478e-b420-238dde9e419c service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:09.702211 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca650001-73c1-478e-b420-238dde9e419c service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:09.707787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17bc88-1113-4ebd-95ea-1d27bc0358fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:09.708650 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17bc88-1113-4ebd-95ea-1d27bc0358fc service placement] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:09.709800 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17bc88-1113-4ebd-95ea-1d27bc0358fc service placement] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:09.710393 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17bc88-1113-4ebd-95ea-1d27bc0358fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:09.712273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-fb17bc88-1113-4ebd-95ea-1d27bc0358fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:09.725809 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17bc88-1113-4ebd-95ea-1d27bc0358fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:09.727488 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17bc88-1113-4ebd-95ea-1d27bc0358fc service placement] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:09.740787 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1452/2923] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:29:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:29:09.759182 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b5acc-dfd6-43e2-bc94-4d117d80fbf4 service placement] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:09.761157 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-013b5acc-dfd6-43e2-bc94-4d117d80fbf4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:09.778504 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013b5acc-dfd6-43e2-bc94-4d117d80fbf4 service placement] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:09.783822 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:09.784447 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:09.784716 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:09.784989 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] PATH_INFO: `/v3/users/b13d21124dbf41459543426847b67bea` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:09.786126 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b13d21124dbf41459543426847b67bea)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:09.790537 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:09.835245 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b13d21124dbf41459543426847b67bea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:09.843010 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b13d21124dbf41459543426847b67bea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:09.843681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b13d21124dbf41459543426847b67bea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:09.844306 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b13d21124dbf41459543426847b67bea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:09.851322 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b13d21124dbf41459543426847b67bea'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:09.868667 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1472/2924] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:29:09 2022] DELETE /identity/v3/users/b13d21124dbf41459543426847b67bea => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:29:10.046436 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17bc88-1113-4ebd-95ea-1d27bc0358fc service placement] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:10.062783 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17bc88-1113-4ebd-95ea-1d27bc0358fc service placement] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:10.112050 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=100 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:29:10.120883 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:10.121506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:10.121781 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:10.122048 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] PATH_INFO: `/v3/projects/b4ebb6e9554f4c0d985bf90411a4e36f` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:10.123128 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4ebb6e9554f4c0d985bf90411a4e36f)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:10.125068 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:10.170414 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4ebb6e9554f4c0d985bf90411a4e36f', 'request_id': 'req-08704ae4-8cc9-46bd-82e4-c84459720b01'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.177144 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4ebb6e9554f4c0d985bf90411a4e36f', 'request_id': 'req-08704ae4-8cc9-46bd-82e4-c84459720b01'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.182749 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4ebb6e9554f4c0d985bf90411a4e36f', 'request_id': 'req-08704ae4-8cc9-46bd-82e4-c84459720b01'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.193419 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] The token cache is being invalidate because project b4ebb6e9554f4c0d985bf90411a4e36f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:29:10.193700 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.194318 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.195807 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1453/2925] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:29:10 2022] DELETE /identity/v3/projects/b4ebb6e9554f4c0d985bf90411a4e36f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:29:10.200631 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:10.202045 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:10.215584 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5350261-65cf-4f96-9ded-497bc752295f admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:10.264142 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:29:10.270992 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:10.272021 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:10.272400 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:10.272754 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] PATH_INFO: `/v3/users/4e94dd8f024c4e6487f89481c614eb27` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:10.274141 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e94dd8f024c4e6487f89481c614eb27)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:10.277075 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:10.323340 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e94dd8f024c4e6487f89481c614eb27'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.330488 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e94dd8f024c4e6487f89481c614eb27'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.331327 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e94dd8f024c4e6487f89481c614eb27'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.332198 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e94dd8f024c4e6487f89481c614eb27'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.340158 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-8abaa7c7-46af-48d1-95a4-dd83cfb14e34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e94dd8f024c4e6487f89481c614eb27'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.357458 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1473/2926] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:29:10 2022] DELETE /identity/v3/users/4e94dd8f024c4e6487f89481c614eb27 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:29:10.534674 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:10.536186 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:10.550367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08704ae4-8cc9-46bd-82e4-c84459720b01 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:10.606283 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:10.606913 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:10.607193 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:10.607490 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] PATH_INFO: `/v3/projects/f53c623bc71a487cad8d53dc5c66b349` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:10.608603 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f53c623bc71a487cad8d53dc5c66b349)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:10.610774 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:10.656260 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f53c623bc71a487cad8d53dc5c66b349', 'request_id': 'req-801f02de-255c-4c1a-bdfe-a40904784d52'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.662853 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f53c623bc71a487cad8d53dc5c66b349', 'request_id': 'req-801f02de-255c-4c1a-bdfe-a40904784d52'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.668889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f53c623bc71a487cad8d53dc5c66b349', 'request_id': 'req-801f02de-255c-4c1a-bdfe-a40904784d52'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.679506 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] The token cache is being invalidate because project f53c623bc71a487cad8d53dc5c66b349 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:29:10.679794 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.680420 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:29:10.682551 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1454/2927] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:29:10 2022] DELETE /identity/v3/projects/f53c623bc71a487cad8d53dc5c66b349 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:29:12.685284 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350446f-cd22-41a8-bc65-9a63b067154b None None] REQUEST_METHOD: `POST` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:12.685802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350446f-cd22-41a8-bc65-9a63b067154b None None] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:12.685802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3350446f-cd22-41a8-bc65-9a63b067154b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:12.704718 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.auth.core [None req-3350446f-cd22-41a8-bc65-9a63b067154b None None] MFA Rules not processed for user `f4638a1936f5467db502bda82320b113`. Rule list: `[]` (Enabled: `True`). {{(pid=81589) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 10 14:29:12.705827 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-3350446f-cd22-41a8-bc65-9a63b067154b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:12.759080 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1474/2928] 10.222.0.26 () {58 vars in 958 bytes} [Sat Sep 10 14:29:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 10 14:29:13.075573 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:13.076974 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:13.092204 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:29:13.135843 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f02de-255c-4c1a-bdfe-a40904784d52 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:13.139371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f5852-b309-4982-a2a9-12cafd339247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:13.140023 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f5852-b309-4982-a2a9-12cafd339247 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:13.140300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f5852-b309-4982-a2a9-12cafd339247 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:13.140570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f5852-b309-4982-a2a9-12cafd339247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:13.142273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f65f5852-b309-4982-a2a9-12cafd339247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:13.152154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f5852-b309-4982-a2a9-12cafd339247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:13.153567 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65f5852-b309-4982-a2a9-12cafd339247 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:13.161477 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1455/2929] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:29:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:29:22.752660 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=52,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:29:23.157865 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=20 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:29:30.325056 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350446f-cd22-41a8-bc65-9a63b067154b None None] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:30.334681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3350446f-cd22-41a8-bc65-9a63b067154b None None] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:30.338926 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6c93a-7717-4f33-9516-408926f373c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:30.339552 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c93a-7717-4f33-9516-408926f373c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:30.339831 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c93a-7717-4f33-9516-408926f373c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:30.340101 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e6c93a-7717-4f33-9516-408926f373c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:30.341801 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-15e6c93a-7717-4f33-9516-408926f373c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:30.405412 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6c93a-7717-4f33-9516-408926f373c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:30.406859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e6c93a-7717-4f33-9516-408926f373c8 service neutron] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:30.416464 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1475/2930] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:29:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:29:40.412041 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=30 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:29:52.416546 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f5852-b309-4982-a2a9-12cafd339247 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:29:52.418555 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-f65f5852-b309-4982-a2a9-12cafd339247 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:52.429273 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65f5852-b309-4982-a2a9-12cafd339247 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:29:52.433300 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91b9bb-8eab-49b0-be14-1e6844097bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:29:52.433901 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91b9bb-8eab-49b0-be14-1e6844097bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:29:52.434176 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91b9bb-8eab-49b0-be14-1e6844097bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:29:52.434443 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca91b9bb-8eab-49b0-be14-1e6844097bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:29:52.436132 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-ca91b9bb-8eab-49b0-be14-1e6844097bb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:29:52.450120 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91b9bb-8eab-49b0-be14-1e6844097bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:29:52.451617 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca91b9bb-8eab-49b0-be14-1e6844097bb9 service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:29:52.459735 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1456/2931] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:29:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:30:02.456005 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=6 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:31:22.031155 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6c93a-7717-4f33-9516-408926f373c8 service neutron] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:31:22.033512 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-15e6c93a-7717-4f33-9516-408926f373c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:31:22.121503 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e6c93a-7717-4f33-9516-408926f373c8 service neutron] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:31:22.127678 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f75e6-1884-49cf-94e3-f494c64b8742 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb24a1b55ee7408ab24c68991aec54ec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:31:22.128283 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f75e6-1884-49cf-94e3-f494c64b8742 service nova] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:31:22.128613 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f75e6-1884-49cf-94e3-f494c64b8742 service nova] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:31:22.128891 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679f75e6-1884-49cf-94e3-f494c64b8742 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:31:22.130988 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-679f75e6-1884-49cf-94e3-f494c64b8742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:31:22.145355 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f75e6-1884-49cf-94e3-f494c64b8742 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:31:22.146962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679f75e6-1884-49cf-94e3-f494c64b8742 service nova] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:31:22.159732 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1476/2932] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:31:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 10 14:31:32.154443 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=34 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:31:34.293791 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91b9bb-8eab-49b0-be14-1e6844097bb9 service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:31:34.302627 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca91b9bb-8eab-49b0-be14-1e6844097bb9 service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:31:34.306743 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2bf43-65bf-4a7f-aed4-543471179a8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '12c71caf31354ec193e3e2da231f1e77', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:31:34.307329 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2bf43-65bf-4a7f-aed4-543471179a8a service neutron] REQUEST_METHOD: `GET` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:31:34.307646 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2bf43-65bf-4a7f-aed4-543471179a8a service neutron] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:31:34.307920 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2bf43-65bf-4a7f-aed4-543471179a8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:31:34.309630 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-4bc2bf43-65bf-4a7f-aed4-543471179a8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:31:34.372557 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2bf43-65bf-4a7f-aed4-543471179a8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:31:34.374016 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2bf43-65bf-4a7f-aed4-543471179a8a service neutron] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:31:34.384311 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1457/2933] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:31:40.689853 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f75e6-1884-49cf-94e3-f494c64b8742 service nova] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:31:40.691582 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-679f75e6-1884-49cf-94e3-f494c64b8742 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:31:40.779266 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679f75e6-1884-49cf-94e3-f494c64b8742 service nova] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:31:40.783918 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:31:40.784540 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:31:40.784962 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:31:40.785232 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] PATH_INFO: `/v3/users/f4638a1936f5467db502bda82320b113` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:31:40.786350 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4638a1936f5467db502bda82320b113)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:31:40.788349 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:31:40.834679 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4638a1936f5467db502bda82320b113'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:40.842942 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4638a1936f5467db502bda82320b113'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:40.843625 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4638a1936f5467db502bda82320b113'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:40.844223 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4638a1936f5467db502bda82320b113'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:40.851247 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4638a1936f5467db502bda82320b113'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:40.870575 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1477/2934] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:31:40 2022] DELETE /identity/v3/users/f4638a1936f5467db502bda82320b113 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:31:41.046946 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2bf43-65bf-4a7f-aed4-543471179a8a service neutron] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:31:41.103277 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2bf43-65bf-4a7f-aed4-543471179a8a service neutron] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:31:41.158208 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:31:41.158827 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:31:41.159104 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:31:41.159371 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] PATH_INFO: `/v3/projects/c8d41b873ac94a33beb63a2d89a218ff` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:31:41.160518 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8d41b873ac94a33beb63a2d89a218ff)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:31:41.162402 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:31:41.207565 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8d41b873ac94a33beb63a2d89a218ff', 'request_id': 'req-c29ff366-8dfd-46c1-b1b2-7d8584176056'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.215326 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8d41b873ac94a33beb63a2d89a218ff', 'request_id': 'req-c29ff366-8dfd-46c1-b1b2-7d8584176056'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.216570 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:31:41.222284 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8d41b873ac94a33beb63a2d89a218ff', 'request_id': 'req-c29ff366-8dfd-46c1-b1b2-7d8584176056'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.234458 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] The token cache is being invalidate because project c8d41b873ac94a33beb63a2d89a218ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:31:41.235001 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.235661 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.237154 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1458/2935] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:31:41 2022] DELETE /identity/v3/projects/c8d41b873ac94a33beb63a2d89a218ff => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:31:41.241302 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:31:41.242681 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:31:41.255541 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f709d6-4c52-48b9-af9e-b420d9071429 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:31:41.311069 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:31:41.311733 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:31:41.312012 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:31:41.312285 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] PATH_INFO: `/v3/users/26c4e1d1e313497cbfa63825c59c724d` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:31:41.313406 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26c4e1d1e313497cbfa63825c59c724d)` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:31:41.315330 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:31:41.327538 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:31:41.362336 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26c4e1d1e313497cbfa63825c59c724d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.369436 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26c4e1d1e313497cbfa63825c59c724d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.370053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26c4e1d1e313497cbfa63825c59c724d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.370655 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26c4e1d1e313497cbfa63825c59c724d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.377528 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.notifications [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26c4e1d1e313497cbfa63825c59c724d'} {{(pid=81589) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.395196 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1478/2936] 10.222.0.26 () {58 vars in 1224 bytes} [Sat Sep 10 14:31:41 2022] DELETE /identity/v3/users/26c4e1d1e313497cbfa63825c59c724d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:31:41.569367 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] Authenticating user token {{(pid=81588) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:31:41.571473 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.fernet_utils [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81588) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:31:41.586174 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29ff366-8dfd-46c1-b1b2-7d8584176056 admin admin] Validating token access rules against request {{(pid=81588) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:31:41.647041 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '25a17c1158c94c229e16c77c070bd9d7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7a00398c85824c23b37ffe5fee90b840', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81588) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:31:41.647744 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:31:41.648026 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] SCRIPT_NAME: `/identity` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:31:41.648319 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] PATH_INFO: `/v3/projects/0aa91e4e0c684befa53d7f0fc50ac25a` {{(pid=81588) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:31:41.649552 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aa91e4e0c684befa53d7f0fc50ac25a)` {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:31:41.651523 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] RBAC: Authorization granted {{(pid=81588) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:31:41.696997 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aa91e4e0c684befa53d7f0fc50ac25a', 'request_id': 'req-26cbc37f-531b-4fe0-b767-e7993efe334f'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.704275 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aa91e4e0c684befa53d7f0fc50ac25a', 'request_id': 'req-26cbc37f-531b-4fe0-b767-e7993efe334f'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.709754 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aa91e4e0c684befa53d7f0fc50ac25a', 'request_id': 'req-26cbc37f-531b-4fe0-b767-e7993efe334f'} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.721867 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] The token cache is being invalidate because project 0aa91e4e0c684befa53d7f0fc50ac25a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81588) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 10 14:31:41.722151 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.722889 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG keystone.notifications [None req-26cbc37f-531b-4fe0-b767-e7993efe334f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81588) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 10 14:31:41.724384 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: [pid: 81588|app: 0|req: 1459/2937] 10.222.0.26 () {58 vars in 1233 bytes} [Sat Sep 10 14:31:41 2022] DELETE /identity/v3/projects/0aa91e4e0c684befa53d7f0fc50ac25a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 10 14:31:51.391813 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=22,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:31:51.718857 dv-cinder-856901-1-iscsi devstack@keystone.service[81588]: DEBUG dbcounter [-] [81588] Writing DB stats keystone:DELETE=9,keystone:SELECT=56,keystone:INSERT=1 {{(pid=81588) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 10 14:32:53.973653 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] Authenticating user token {{(pid=81589) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 10 14:32:53.975053 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:32:54.080385 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6845e9d-a51c-43f0-a4d8-5b9a5efb8414 admin admin] Validating token access rules against request {{(pid=81589) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 10 14:32:54.142859 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5171671b-8600-479c-bf23-71d6250595bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2a6a70a10cfa49348cc8b2b42870909d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '337395b94d7c4530a3fb24594976c6ca', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81589) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 10 14:32:54.143525 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171671b-8600-479c-bf23-71d6250595bf service placement] REQUEST_METHOD: `GET` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 10 14:32:54.143802 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171671b-8600-479c-bf23-71d6250595bf service placement] SCRIPT_NAME: `/identity` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 10 14:32:54.144070 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5171671b-8600-479c-bf23-71d6250595bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81589) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 10 14:32:54.145768 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.fernet_utils [None req-5171671b-8600-479c-bf23-71d6250595bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81589) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 10 14:32:54.157414 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171671b-8600-479c-bf23-71d6250595bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 10 14:32:54.158982 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5171671b-8600-479c-bf23-71d6250595bf service placement] RBAC: Authorization granted {{(pid=81589) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 10 14:32:54.168908 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: [pid: 81589|app: 0|req: 1479/2938] 10.222.0.26 () {60 vars in 1365 bytes} [Sat Sep 10 14:32:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 10 14:33:04.164475 dv-cinder-856901-1-iscsi devstack@keystone.service[81589]: DEBUG dbcounter [-] [81589] Writing DB stats keystone:SELECT=74 {{(pid=81589) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}